TurnSignal

Is TurnSignal safe to add to CI? Security and data

In short

  • The reporter never changes Playwright's exit code, and if our API is down your tests run as usual.
  • TurnSignal never receives your repository or source code. Test source files are left out of uploaded traces.
  • Secrets in errors and traces are redacted by pattern in your CI before upload. Screenshots and videos can't be redacted; you choose whether to send them.
  • Not for regulated data yet: no BAA or HIPAA terms, no self-hosted version. Use it with tests that run on synthetic data.
  • Test runs are deleted after 90 days by default, and you can delete everything yourself at any time.

What runs in your CI

  • The turnsignal reporter runs inside your Playwright run, next to your other reporters. It never changes Playwright's exit code: your CI passes or fails exactly as it would without TurnSignal.
  • Results are written to disk in CI first and sent while the tests run. If our API is unreachable, the reporter gives up quickly and npx turnsignal upload sends the saved results later.
  • It needs one secret, the project token (TURNSIGNAL_TOKEN), which works only for that one project. It needs no access to your repository.

How secrets are kept out

  • Before anything is written to disk or sent, error messages, stacks and code snippets are scanned and credentials are replaced with [REDACTED]: Authorization headers, passwords in URLs, password=/api_key=-style values, whole Cookie headers, webhook URLs (Slack, Discord), common token formats (GitHub, GitLab, npm, AWS, OpenAI, Anthropic, Stripe, Google, Azure, JWTs, private keys), and the values of your CI's secret-looking environment variables (*TOKEN*, *SECRET*, *PASSWORD*, *KEY*). Errors are checked again on our servers.
  • Playwright traces are redacted in CI before upload: sensitive headers and cookies are blanked and secret patterns are replaced in the network log, console output and text resources. A trace that cannot be redacted safely is not uploaded.
  • You can add your own patterns with the reporter's redact option or TURNSIGNAL_REDACT.
  • Redaction matches known patterns, so it can miss an unusual secret. Commit messages, test titles, screenshots and videos are not redacted. Choose what to send with TURNSIGNAL_ARTIFACTS=trace,screenshot (or none); project owners can also turn artifacts off.
  • Background: Secrets in Playwright traces.

What TurnSignal receives and stores

  • Test names and file paths, outcomes, durations and attempts, error messages and stack traces (after redaction), branch, commit id and message, the commit author's name (turn off with TURNSIGNAL_SEND_AUTHOR=0), the CI job link, the runner's hostname and memory use.
  • Screenshots, videos and traces, only if your Playwright config records them and your project keeps them.
  • Never your source code, payment details or IP addresses in our database; no analytics or advertising trackers.
  • Data travels over TLS. The database (Neon, Postgres on AWS us-east-1) and our artifact storage are encrypted at rest. The full list of providers is in the privacy notice.

What you can turn off, and what always leaves CI

  • Always sent, because the verdict is built from them: test titles and file paths, outcomes and durations, error messages, stacks and the code lines where a test failed (after secret redaction), branch, commit id and commit subject. Turning artifacts off does not stop these.
  • TURNSIGNAL_ARTIFACTS=none sends no screenshots, videos or traces (or list the kinds you want: trace,screenshot). Project owners can also switch artifacts off for a project.
  • TURNSIGNAL_REDACT adds your own patterns (for example an internal id format). They apply to errors and traces. Redaction is built for credentials: it can't reliably find names or other free-text personal data.
  • TURNSIGNAL_SEND_AUTHOR=0 leaves out the commit author. Organization owners can store screenshots, videos and traces in their own S3-compatible bucket instead of ours (Members page).

Health, payment and other regulated data

  • TurnSignal is a hosted public beta. We do not sign business associate agreements (BAA) or offer HIPAA terms, and there is no self-hosted version.
  • So don't send results from tests whose screens or errors contain real patients' or members' data, card data or similar: a failed toHaveText on such a page puts that data in the error message.
  • Tests that run on synthetic data are fine, and so are the parts of TurnSignal that upload nothing (npx turnsignal governance and npx turnsignal analyze run only in your CI).

Retention and deletion

  • Test runs are deleted automatically after 90 days, or the retention period set for your project.
  • Screenshots, videos and traces: 30 days for failed and flaky tests, 7 days for passed tests, and always with their run.
  • You can delete projects, organizations and your account yourself; the data goes right away.
  • Questions or a copy of your data: email hello@turnsignal.ai.

If TurnSignal shuts down

  • If we ever shut TurnSignal down, we give at least 60 days' notice by email to organization owners and on turnsignal.ai, and every organization can get a full export of its runs and test history before anything is deleted: the JSON export in the dashboard, and on request a complete export of every test result.
  • Organization owners can download the organization's data (members, projects, runs, artifact list) as JSON from the dashboard at any time.