Privacy

What this site collects

trolleybench is a benchmark of language models. It has no accounts and does not ask who you are. This page says what is stored, where, and for what. Last updated 2026-11-01.

In short

  • Your answers in Play and the Workbench stay in your browser. They are not sent to this site.
  • Your API keys never reach this site. The browser runner talks to the model provider directly.
  • Page views are counted with Google Analytics. That is the only third-party tracker.
  • Hosted runs are public. When a model is run through the API, MCP or the browser runner and submitted, its answers are stored and shown next to its name.
  • There are no accounts, no email collection, no ads and no sale of data.

Kept in your browser

The site uses your browser’s localStorage for conveniences that never leave your device:

  • Your Play answers, so the comparison with people and models can be shown after the last dilemma and survives a reload.
  • Your Workbench picks and the models you highlight on the results page.
  • Your theme and accent colour.
  • A resume link for an unfinished browser run (the run’s id, its token and the model name), in sessionStorage, which the browser drops when you close the tab. Your API key is not part of it.

Clear your site data in the browser to remove all of it. If your browser blocks storage the site still works; it just forgets these choices.

Analytics

Every page loads Google Analytics 4 (measurement ID G-M3QHMNTN74) so the maintainers can see which pages are read. The site sends a page view for each route you open, with its path, title and address. It does not send your Play answers, your API keys or anything you type into a form.

Google sets cookies (such as _ga) and receives your IP address and browser details as part of that request, under Google’s privacy policy. GA4 does not keep IP addresses in its reports. The site does not currently ask for consent before loading the tag. See your choices for how to opt out.

Fonts are served from this site, not from Google, so loading them tells Google nothing.

Hosted runs (API, MCP, browser runner)

These features test a model, not you. When one is used, the database stores:

  • Each answer as it arrives: the prompt, the reply text, how it was scored, and timing.
  • The model name you give, marked as self-reported.
  • A salted hash of your IP address. It is used to rate-limit runs (20 an hour) and to group one submitter’s sessions together. The address itself is not stored with the run, the hash is never shown, and on the site a submitter appears only as “community N”.

Results are public. Do not put anything in a model name, a system prompt or a reply that you would not want published. Maintainers review submitted runs, and a run can be hidden or removed on request (see contact).

Hosting and logs

The site runs on Vercel and its database is on Neon. Like any host, they process requests and may keep short-lived server logs that include IP addresses and request paths. This site does not read those logs for anything beyond debugging and abuse prevention.

API keys

A key you type into the browser runner is held in the page’s memory and sent only to the provider you chose. It is not sent to this site, not written to storage and not logged. The API and MCP routes never involve a key. Use a key with a spend limit, and revoke it afterwards if you share your screen.

Your choices

  • Opt out of analytics with Google’s browser add-on, or by blocking googletagmanager.com in your browser or content blocker. The site works the same without it.
  • Remove local data by clearing this site’s storage in your browser.
  • Ask for a run to be removed. Open an issue with the run’s address and we will hide or delete it. Because the stored hash cannot be reversed to an address, we cannot look up “your” runs for you; send the run links.
  • Run it yourself. The CLI works fully offline with no database, and nothing leaves your machine except calls to the model providers you configure.

Contact and changes

Questions or removal requests: open an issue at github.com/Sherlemious/trolleybench/issues. Do not post secrets there. If this page changes, the date at the top changes and the history is in the repository.

This is a plain-language description of how the site works, not legal advice.