Feature flags in a CI pipeline

I&#x27;ve done something like this with CircleCI and Github labels. The Continuation orb[1] allows you to dynamically generate your CI yml rather than using a static workflow. A Python script is executed which uses information about the PR (author, timestamp, labels applied to the PR, etc) and a CI template yml to add, remove, or skip jobs from the workflow. Some use cases are:<p><pre><code> - Replacing a job with a newer job for testing purposes, as described in the article. - Opting out of jobs, like &quot;skip-deploy&quot; or &quot;skip-unit&quot; labels, to speed up CI in some cases when these jobs aren&#x27;t needed. - Making some optional jobs required depending on the files that have been modified in the PR. You can get a list of modified files in a PR using the Github API. We use this for example if a new feature necessitates a new CI job but you want this job to be optional for branches that don&#x27;t have this feature code yet. </code></pre> The additional complexity can make this a non-starter for some teams. In our case these use cases are more convenient for us than confusing for the eng team.<p>1: <a href="https:&#x2F;&#x2F;circleci.com&#x2F;developer&#x2F;orbs&#x2F;orb&#x2F;circleci&#x2F;continuation" rel="nofollow">https:&#x2F;&#x2F;circleci.com&#x2F;developer&#x2F;orbs&#x2F;orb&#x2F;circleci&#x2F;continuatio...</a>

It looks like an additional tool and a new remote service to talk to. Are there any CIs that don&#x27;t support editable variables on scopes larger than a build?<p>The article talks about GH Actions, they have org&#x2F;repo&#x2F;env variables.<p>Even Jenkins has variables usable from the build.

We add a &quot;branch config&quot; JSON file for each branch, which is later merged with some CUE. We put metadata and &quot;feature flag&quot; like values in there. This full config value is passed to all subjobs and makes it easy to change and grow our CI.<p>The main problem I see with real feature flags in CI is that it can make reproducing a build nearly impossible. What happens if someone flips a flag in the middle of a build? (where that flag value is checked in multiple places, so the flip happens in between the checks)

Flagon&#x27;s exit codes are as follows:<p>0 the flag queried is on (true) 1 the flag queried is off (false) 1 an error occurred querying the flag<p>I think that it would be nice that error and false had different exit codes.

I&#x27;ve been thinking about a way to add feature flags to my project&#x27;s CI pipeline. What makes me afraid of adding such thing is reliability: if someone breaks the feature flag service, now the CI pipeline is broken too. I could just give them a default value if the communication fails, but then we will have builds that are skipping checks potentially due to transient errors. I would be very curious to know the guidance for using this tool in my case.

I’ve been looking for a way to do integration tests for our pipelines. So far haven’t found anything I could use.