GitHub Actions for Open Source Projects

You'll need

GitHub Actions Setup complete
your repository accepts pull requests from forks
Editor access to the Flakiness.io project

You'll have

reports from pull requests from forks uploading to Flakiness.io from your regular test workflow.

Why fork PRs are different

GitHub Actions doesn’t give repository secrets or OIDC tokens to workflows triggered by pull_request events from forks, so that a malicious fork can’t print them to the workflow log. Without either, the reporter has no way to authenticate its upload.

The flakiness/authenticate-fork-prs action gets these jobs an upload token instead.

Setup

Allow uploads from forks. On the project’s Access page, under Pull Requests from Forks, turn on Allow uploads from forks. Only project Editors see this section.
Add the action to your workflow, before the step that runs your tests:
.github/workflows/tests.yml
```
permissions:
  contents: read
  actions: read # Only needed for private repositories

steps:
  - uses: actions/checkout@v4
  - uses: flakiness/authenticate-fork-prs@v1
    with:
      flakiness-project: my-org/my-project
  - name: Run tests
    run: npx playwright test
```
Replace my-org/my-project with your organization and project slugs. The Access page shows this snippet with them filled in.

In a private repository, the job’s token needs actions: read so that Flakiness.io can find the action’s artifact. Public repositories don’t need it.

The action sets FLAKINESS_ACCESS_TOKEN for the steps that follow, and the reporter uploads with it.

How it works

In a pull request from a fork, the action:

picks a random secret and uploads an artifact named after the secret’s SHA-256 hash into its own workflow run;
sends the secret, the run ID, and the job’s GITHUB_TOKEN to Flakiness.io;
receives an upload token in return.

Flakiness.io lists the artifacts of that run in the project’s linked repository, using the job’s GITHUB_TOKEN. It issues the token only if the run belongs to a pull request from a fork and has an artifact named after the secret’s hash. Only a job of that run could have uploaded the artifact, and only that job knows the secret. Flakiness.io doesn’t store the GITHUB_TOKEN.

The upload token expires after 6 hours. It can upload reports to the project, and otherwise has the same access as an anonymous visitor: it can read a public project, and nothing of a private one.

Security considerations

Anyone who can open a pull request can upload reports. The code of a pull request from a fork belongs to its author, so the reports it uploads can contain anything. GitHub’s approval setting for fork pull request workflows decides whose pull requests run workflows without a maintainer’s approval.
Turning off Allow uploads from forks takes effect immediately. Tokens that were already issued stop working too.
Changes to the setting are recorded in the audit log.