products

Test Feature Flags Using Cypress and Flagsmith

How to control the web application feature flags during end-to-end tests.

This blog post teaches you how to control the feature flags provided by 3rd party services like Flagsmith, LaunchDarkly, Split, etc during end-to-end tests.

I have created a small project on Flagsmith.com and added a single feature flag "feature_a". At first the flag is turned off.

Feature A toggle on Flagsmith

My web application uses the Flagsmith JavaScript SDK to fetch the flags at runtime. Depending on the feature flag presence, the application renders a different message.

🎁 You can find the application code and the Cypress tests in the repo bahmutov/flagsmith-cypress-example.

The index.html loads the Flagsmith library and the application code

public/index.html
1
2
3
4
5
6
7
8
9
10
11
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>flagsmith-cypress-example</title>
  </head>
  <body>
    <script src="https://cdn.jsdelivr.net/npm/flagsmith/index.js"></script>
    <script src="app.js"></script>
  </body>
</html>

The application code in app.js inserts an element with the text determined by the feature flag.

public/app.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// output DIV element
const div = document.createElement('div')
div.setAttribute('id', 'feature-area')
document.body.appendChild(div)

div.appendChild(document.createTextNode('Initializing...'))

function render() {
  const shouldShow = flagsmith.hasFeature('feature_a')
  const label = (shouldShow ? 'Showing' : 'Not showing') + ' feature A'
  div.replaceChild(document.createTextNode(label), div.firstChild)
}

// https://docs.flagsmith.com/clients/javascript/
flagsmith.init({
  // comes from the Flagsmith project settings
  environmentID: 'gxzgHaQ84gijocUvctHJFb',
  onChange: render,
})

By default the feature_a is turned off. Thus the application sets the label to "Not showing feature A".

When the Feature A is off

If we flip the feature switch and reload the web page, the feature A will be active and the label changes.

Activate Feature A

Let's test the web application behavior using Cypress test runner. I want to confirm the following three scenarios:

  • the application is showing the loading message while the feature flags are fetched
  • the application is working correctly when Feature A is turned OFF during the test
  • the application is working correctly when Feature A is turned ON during the test

The loading message

Let's load the page from a Cypress test to see what is going on. I will flip the feature A back to "off" in the Flagmisth project. The test in the spec.js uses the cy.visit command to load the page.

cypress/integration/spec.js
1
2
3
4
5
6
7
/// <reference types="cypress" />

describe('Flagsmith Cypress example', () => {
  it('loads the page', () => {
    cy.visit('/')
  })
})

Notice the Command Log showing the Ajax call the Flagsmith SDK is making to its API endpoint to fetch the current flags.

Flagsmith SDK is fetching the features

Click on that network call to dump its contents in the DevTools console. The server response has all the feature flags in an array.

The response includes our feature A object

In the future tests we can control the response value, but for now let's just slow the network call to make the loading message visible during the test.

cypress/integration/spec.js
1
2
3
4
5
6
7
8
9
10
11
it('shows the loading message', () => {
  // slow down the network call by 1 second
  cy.intercept('/api/v1/flags/', (req) =>
    Cypress.Promise.delay(1000).then(req.continue),
  ).as('flags')
  cy.visit('/')
  cy.contains('#feature-area', 'Initializing...').should('be.visible')
  // wait for the feature flags Ajax call
  cy.wait('@flags')
  cy.contains('Initializing...').should('not.exist')
})

The test proves the loading message is visible at first, then it goes away.

The loading message test

Tip: read the blog post Be Careful With Negative Assertions for a detailed essay about testing the loading element.

Test application with Feature A turned OFF

Stub the features Ajax call

Let's test how our application behaves without the feature A. We already have the network call response from the Flagsmith API - copy the response object body from the Network tab of the browser's DevTools.

Copy the feature flags API call response body from the Network tab

Save the text as a JSON file in cypress/fixtures/no-feature-a.json.

cypress/fixtures/no-feature-a.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[
  {
    "id": 56756,
    "feature": {
      "id": 10804,
      "name": "feature_a",
      "created_date": "2021-07-15T23:38:27.661659Z",
      "description": "The first feature",
      "initial_value": null,
      "default_enabled": false,
      "type": "STANDARD"
    },
    "feature_state_value": null,
    "enabled": false,
    "environment": 9128,
    "identity": null,
    "feature_segment": null
  }
]

The test can mock the Ajax call using the above fixture file using the cy.intercept command.

1
2
3
4
5
6
it('does not show feature A', () => {
  cy.intercept('/api/v1/flags/', { fixture: 'no-feature-a.json' }).as('flags')
  cy.visit('/')
  cy.wait('@flags')
  cy.contains('#feature-area', 'Not showing feature A')
})

Stub the Flagsmith Ajax call test

Modify the response only

Stubbing the entire Flagsmith call seems excessive. What if there are a lot of features? Do we have to constantly update the fixture file? We are only interested in the feature_a flag. Let's spy on the Ajax call to /api/v1/flags/ and just modify the response to always have the feature_a OFF.

Tip: Cypress bundles Lodash library as Cypress._ so you can use its powerful utility functions from tests.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
it('does not show feature A (modify response)', () => {
  cy.intercept('/api/v1/flags/', (req) => {
    req.continue((res) => {
      expect(res.body, 'response is a list of features').to.be.an('array')
      const featureA = Cypress._.find(
        res.body,
        (f) => f.feature.name === 'feature_a',
      )
      // make sure the feature is present
      expect(featureA, 'feature_a is present').to.be.an('object')
      expect(featureA).to.have.property('enabled')
      console.log(
        'changing %s from %s to %s',
        featureA.feature.name,
        featureA.enabled,
        false,
      )
      featureA.enabled = false
    })
  }).as('flags')
  cy.visit('/')
  cy.wait('@flags')
  cy.contains('#feature-area', 'Not showing feature A')
})

The above test uses req.continue((res) => { ... }) callback to inspect the response from the Flagsmith server and modify the feature flag A. By sprinkling a few assertions there we verify that the flag is still present. The console log message shows that even if we toggle the feature flag for the project, the test still overrides it.

Override the feature_a flag in the response object

Test application with Feature A turned ON

Similar to the previous test, we can verify the application's behavior when the feature A is turned ON.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
it('shows the feature A', () => {
  cy.intercept('/api/v1/flags/', (req) => {
    req.continue((res) => {
      expect(res.body, 'response is a list of features').to.be.an('array')
      const featureA = Cypress._.find(
        res.body,
        (f) => f.feature.name === 'feature_a',
      )
      // make sure the feature is present
      expect(featureA, 'feature_a is present').to.be.an('object')
      expect(featureA).to.have.property('enabled')
      console.log(
        'changing %s from %s to %s',
        featureA.feature.name,
        featureA.enabled,
        true,
      )
      featureA.enabled = true
    })
  }).as('flags')
  cy.visit('/')
  cy.wait('@flags')
  cy.contains('#feature-area', 'Showing feature A').should('be.visible')
})

Override the feature_a flag to be true during the test

Reusable function

Modifying the feature flags can be abstracted into a utility function. For example, we could specify multiple flags to be overwritten, and the utility function will set up a single network intercept with its logic to find and set each feature flag.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
const setFeatureFlags = (flags = {}) => {
  expect(flags).to.be.an('object').and.not.to.be.empty

  cy.intercept('/api/v1/flags/', (req) => {
    req.continue((res) => {
      expect(res.body, 'response is a list of features').to.be.an('array')
      Cypress._.forEach(flags, (value, flagName) => {
        const feature = Cypress._.find(
          res.body,
          (f) => f.feature.name === flagName,
        )
        // make sure the feature is present
        expect(feature, 'feature_a is present').to.be.an('object')
        expect(feature).to.have.property('enabled')

        console.log(
          'changing %s from %s to %s',
          feature.feature.name,
          feature.enabled,
          value,
        )
        feature.enabled = value
      })
    })
  }).as('flags')
}

it('controls the flags', () => {
  setFeatureFlags({ feature_a: true })

  cy.visit('/')
  cy.wait('@flags')
  cy.contains('#feature-area', 'Showing feature A').should('be.visible')
})

I like add an assertion to verify the function's arguments like this

1
expect(flags).to.be.an('object').and.not.to.be.empty

The above assertion is printed to the Command Log allowing the test run video to reflect what feature flags were set during the test.

Reusable feature flags function

Happy Feature Flags Testing!

See also