products

Testing Content-Security-Policy using Cypress ... Almost

How to almost test Content-Security-Policy violations in your site using Cypress

I have written about using Content-Security-Policy (usually shortened to just CSP) to protect your website from cross-site scripting attacks. Using CSP you can restrict the sources of JavaScript allowed to run on the page, especially disabling the inline JavaScript - and your users are much safer!

But is your protection actually working? Can you verify that the CSP header is set correctly? Can you write a security test that tries to inject a script and gets stopped? Let me show you how to write such test using Cypress. It is almost a perfect way to sprinkle some security testing into your functional end-to-end tests. I am saying almost, because this approach almost works.

🧭 You can find the source code for this blog post in the repository bahmutov/test-csp.

IMPORTANT

This feature works in Cypress v12.15+, read the blog post CSP Testing Using Cypress.

The app

Let's first create an application to be attacked using script injection. It is a page with an inline script tag

1
2
3
4
5
6
7
8
9
10
11
12
<body>
  <p>Hi</p>
  <!--
    Inline script. If an attacker can modify the page
    that another user sees in such a way that the attacker's script
    tag runs and shows this alert message, then your page is susceptible to
    script injection attacks
  -->
  <script>
    alert(42)
  </script>
</body>

We can serve the application using any static web server, for example using Express.js

index.js
1
2
3
4
5
6
7
const express = require('express')
const app = express()
app.use(express.static('public'))
const port = 3003
app.listen(port, () => {
  console.log(`Server listening at http://localhost:${port}`)
})

When we load this page in the browser we see the inline script executing

Inline script executing on page load

If we see this popup on the page, that's bad. That means an attacker can also somehow inject something (potentially) that would also run, stealing another user's private and sensitive information. Let's stop all inline scripts from running.

Content security policy

Let's lock our web application by serving it with Content-Security-Policy header by using Helmet module. Just add it to our application and restart the server.

index.js
1
2
3
4
5
const express = require('express')
const helmet = require('helmet')
const app = express(
app.use(helmet()) // use defaults
app.use(express.static('public'))

When we see the page headers now we will see a slew of response headers making the page a lot more secure.

Security headers added by Helmet middleware

And the best result - the alert message box does not pop up anymore. Instead we see an angry message in the browser's console

Browser does not execute scripts that are not allowed by the CSP

Great, our Content-Security-Policy is protecting our site's users by blocking all inline scripts.

Reporting violations

The Content-Security-Policy stops the scripts that violate the security policy from running. The browser also prints the security error to the console. We can also report any security errors to an external server using /report-uri option in the policy. For this, we need to actually provide our own CSP policy plus reportUri address to send all attack events.

index.js
1
2
3
4
5
6
7
8
9
10
11
app.use(
  helmet({
    contentSecurityPolicy: {
      directives: {
        // only allow scripts loaded from the current domain
        defaultSrc: ["'self'"],
        reportUri: ['/security-attacks'],
      },
    },
  }),
)

When loading the page we can see the attack reported to the (non-existent) endpoint http://localhost:3003/security-attacks.

Blocked script was reported by the browser

We can inspect the request to see the object sent by the browser with the attack's details

Security violation report fields

Content-Security-Policy and Cypress

Wouldn't it be cool if we could listen to the POST /security-attacks calls from the browser and catch them from Cypress tests to check if the CSP is set up correctly? Using the new cy.route2 API we could rewrite the HTML document to insert script tags from different domains trying to see if we allow executing 3rd party code - and we expect these security attacks to be reported. We could insert script tags directly into the DOM using JS code, again expecting these security violations to be reported.

Well, there is a small problem here. Let's try loading our page in the Cypress browser.

The localhost:3003 loaded in Cypress

Notice that the unsafe inline script has executed - you can see the alert message in the Command Log. Let's inspect the localhost document and list its headers. This is what the browser sees when Cypress visits the page.

The response headers the browser sees when Cypress visits the page

If you compare these headers to the full set of headers on the document in the "normal" browser that I have shown at the start of the blog, you will notice that some security headers are automatically stripped by Cypress. This is done to allow Cypress to load the site in an iframe (thus stripping X-Frame-Options header) and to inject a tiny script at the start of the page (thus it needs to remove the Content-Security-Policy header)

The script injected by Cypress proxy into the visited page

This tiny script that sets document.domain = 'localhost'; is the key to how Cypress works - it allows the JavaScript from the test domain (usually something like localhost:<random port>) to access document from any other domain loaded in an iframe. But to inject this script we need to remove the CSP header, since it would normally prevent unsafe inline scripts.

Attempt 1 - move CSP header into meta

There is a nice little workaround we can perform to still use CSP protection, yet allow Cypress to load the page. The Content-Security-Policy can be set either via a response header (recommended), or via <meta ...> HTML tag (not recommended). In our test, we can copy the header into the document's <HEAD>!

cypress/integration/spec.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
it('loads', () => {
  // we are only interested in the root document
  const url = Cypress.config('baseUrl') + '/'
  cy.route2('/', (req) => {
    if (req.url === url) {
      return req.reply((res) => {
        const csp = res.headers['content-security-policy']
        // really simply <HEAD> rewriting
        // to only insert the CSP meta tag
        res.body = res.body.replace(
          '<head> </head>',
          `
          <head>
            <meta http-equiv="Content-Security-Policy" content="${csp} ">
          </head>
        `,
        )
      })
    }
  })
  cy.visit('/')
})

The above test spies on the / request, looks at the server's response, and copies the CSP header into the <HEAD><meta ...> tag. Let's look at the result:

CSP copied into the meta tag

We have some wins, and we also have a step back.

First, the wins:

  • The content security policy works. The <script>alert(42)</script> inline block did NOT run. We see an error report and no alert command in the Command Log.
  • Cypress tests still work, even though the little script was injected. Turns out, when CSP is set using a <meta> tag, it applies to the content after the <meta> tag itself. Thus Cypress' own inline tag which comes first is fine. What a lucky break.

CSP meta tag applies only after the test injection script

Second, the loss:

  • as you can see in the browser's console, the /report-uri directive does NOT work when the CSP is set via <META> tag unfortunately (for security reasons). Thus my hope of intercepting the triggered POST /security-attacks calls has been dashed.

Attempt 2 - Content-Security-Policy-Report-Only

In the CSP world, there is the second header we could use to NOT block the unsafe scripts, but to report them only. This header is a nice real world migration strategy that allows you to set the CSP and see if it blocks any of the users before turning it on. We could grab the CSP header and copy it to CSP-Report-Only header when loading the document.

1
2
3
4
return req.reply((res) => {
  const csp = res.headers['content-security-policy']
  res.headers['content-security-policy-report-only'] = csp
})

Unfortunately, in Cypress v5.3.0 we started removing content-security-policy-report-only header to be consistent. Sigh, I am thwarted again.

Future solution (I hope)

So in order for Cypress to work without stripping Content-Security-Policy we should keep the original CSP policy plus inject a permission to load just our Cypress script. This could be done by adding to the list of allowed script sources one more script with a random nonce value.

1
2
3
4
5
6
7
// allow Cypress script injection
const nonce = `nonce-<abc..random>`
const csp = res.headers['content-security-policy']
const parsed = parseCSP(csp)
// push one more script source
parsed.scriptSrc.push(nonce)
res.headers['content-security-policy'] = stringifyCSP(parsed)

later on when injecting Cypress script into <HEAD> element inject it with nonce attribute

1
2
3
<script nonce="<abc..random>" type="text/javascript">
document.domain = 'localhost'; var Cypress = ...
</script>

This is why I have opened #1030 a short time (well, 3 years!) ago. One day I will get to it, and then the security testing against cross-site scripting attacks will become a Cypress feature.