Skip to end of metadata
Go to start of metadata

Managing test data represents a major challenge in system development projects.

Read Evolutionary Database Design from Fowler and Sadalage for some background material to the subject.

Use of test data

Test data is used for:

  1. Unit tests
  2. Integration tests
  3. System tests
  4. Acceptance tests and other manual testing
  5. Load test
  6. Migration tests (deploying new versions of a system on previous installments)
  7. Demo

Categorization of test data types

Try minimize the number of test data sets. Often only two types of test data is sufficient:

  1. A small, compact, well-known test data set that people are familiar with. Should represent all types of variations in the system,
    and the data values should mirror real data (i.e. logical, meaningful values).
  2. Volume-based set, primary for load testing purposes.

For unit testing purposes, there might be a third fragmented type of test set which serves as input data and mock data in tests.
These are implemented in test code, they are informal, and will not be used outside the unit test.
The rest of this article will thus leave out these test data as they are of no importance for the challenge of managing test data.

Loading test data

Mechanisms for loading test data into a system:

  1. By code
  2. By database scripts
  3. By a (live) master database
  4. Export/import

When should you use what ?
Where are the test data stored, and in what type of format ?

Loading frequency

How often should the test data be reloaded ?
Does it depend on the type of environment ?

How to deal with relative test dates - i.e. how to update relative date values to the current date ?

Maintaining test data set

How to keep the test data set intact when the application is changing,
and when developers, testers, business analysts, people in marketing, product owners, etc need new tests ?

Who should be responsible for managing the test data ?

  • A single person (characteristics and roles for this person) ?
  • The dev team/test team ?

How to maintain different versions of each test data set ?

Automating the load task

How to automate this task, tools, error reporting, separate jobs in CI, dependencies ?

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Nov 01, 2010

    Challenges

    TODO: Evaluate common strategies and recommend some best practice?

    • How to test DAO's
      • Spring's AbstractDependencyInjectionSpringContextTests
    • How to speed up tests that uses databases when developing?
      • In-memory database
      • Start, stop, create tables and insert data with maven-plugin
    • How to switch between different databases in development, test/ci and production? (Should be able to switch in run-time)
    • What type of plugins and products are well fit for the task?
    • How to use the IDE?
    • How to structure the DAO layer to make it testable?
      • Patterns (lookup, inheritance from a common BaseDAO, etc.)
      • Anti-patterns
    • What should be processed/filtered/sorted on the DAO layer versus what should be handled by the service layer?
    • Using ORM: testing actual retrieval and storing of data, not just by using the ORM cache.
    • Managing test data

    Available tactics

    • ORM
    • In-memory database (HSQLDB or H2)
    • Wrap each test in a transaction to simplify rollback. (See AbstractTransactionalDataSourceSpringContextTests )
    • Wire dependencies with Spring
    • Populate database with DBUnit
    • Let a group of tests depend on another test
      • All DA-tests depend on testDBConnection (so if this test fails all the other should be skipped)
      • A delete or update statement require that the database already contain the given tuple. The depend relationship might be useful also here.