In 2007 we coined the term timeline to describe the coupling between JigZaw test model and agile development methodology and related concepts as Continuous Integration (CI) and continuous production. In other words, when should a certain group of tests be run and who is responsible for executing them. Timeline builds on Multidimensional Test Categorization and provides a starting point and some guidelines. All projects are different, so finding one timeline that will fit all projects is obviously not possible.
Build pipeline and deployment pipeline are newer terms for the same concepts. See references at the bottom for more resources.
The logical timeline of a project may look something like the following:
- Developer push/commit code
- CI server compile and run tests
- CI server deploys artifacts to artifact repository
- Deployment artifact is installed in production
The steps may be different or more steps can be added. Both before and after each step tests and other verification measures can be applied.
Different actors are involved: Developer, CI server, tester, orchestration tools, etc.
- Initially, run all tests as often as possible. All tests in the same group. All tests run both developer and CI server.
- When adding tests with special characteristics, consider adding a tag. Common test groups: slow, externalDatabase, dataDriven
- If it is inconvenient to run a certain group of tests before developer commit/push his/her code, consider excluding that group from the default test suite. The CI server should still run all tests.
- The CI server should run all tests, but when? After every commit if possible. If this takes too much time, consider running some tests later.
- The build artifacts can be deployed to an environment as part of a build step.
- At some point the artifact will be deployed to production.
Within each sprint
|Developer||pre-commit||Default test suite||mvn clean install|
|CI server||post-commit||Default test suite + tests that require special environment||mvn clean install -DdatabaseX -DjmsServerY|
|CI server||Every hour||Slow tests||mvn clean install -DslowTests|
|CI server||On-successful build||deploy artifacts to artifact repository||mvn deploy|
|CI server||Nightly||Deploy to a test environment &
run data-bound tests using data recorded from production
|Developer||End of sprint, before release||compare technical debt/accidental complexity metrics with metrics from previous release|
|Developer||End of sprint||release||mvn release:prepare & mvn release:perform|
- Deployment Pipeline by Fowler mai 2009
- What Is a Deployment Pipeline? from the book Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation By Jez Humble and David Farley, sept 2010
- The Delivery Pipeline is your DevOps Signature
- Configure and run a delivery pipeline in Jenkins by Paul M. Duvall
- Orchestrating Your Delivery Pipelines with Jenkins