Update strategies

edit

SnapshotSettings.SnapshotUpdateStrategy decides what happens when a snapshot does not match:

StrategyVerified filesTest
SnapshotUpdateStrategy.Disallow (default)Never changedFails
SnapshotUpdateStrategy.OverwriteReplaced by the actual filesFails
SnapshotUpdateStrategy.OverwriteWithoutFailureReplaced by the actual filesPasses
SnapshotUpdateStrategy.MergeToolMerged in a merge tool, which is started for each changed fileFails
SnapshotUpdateStrategy.MergeToolSyncMerged in a merge tool; the test waits for the tool to closeFails

In every case, the actual files are written first. Overwrite and OverwriteWithoutFailure then replace the verified files, delete the verified files the assertion no longer produces, and delete the actual files. The merge tool strategies never delete verified files: the failure message lists the ones that are no longer produced.

A file that cannot be written or deleted, because it is locked or its directory is read-only, is retried a few times. If it still fails, the assertion throws a SnapshotException that lists the files, even with OverwriteWithoutFailure: this strategy hides snapshot differences, not a failed update.

Choose a strategy for a run

The strategy usually depends on who runs the tests rather than on the test. Set the SNAPSHOTTESTING_STRATEGY environment variable to the name of a strategy, ignoring case (Overwrite, mergetool, OVERWRITEWITHOUTFAILURE). It sets the default value of SnapshotUpdateStrategy; a strategy set in code takes precedence. A value that is not the name of a strategy is ignored, and the default stays Disallow.

SNAPSHOTTESTING_STRATEGY=OverwriteWithoutFailure dotnet test

Choose a strategy in code

To change the strategy of every test, set it on SnapshotSettings.Default in a module initializer (see Settings). To change it for one assertion, pass a copy of the settings:

var value = new { Name = "John", Age = 42 };
var settings = SnapshotSettings.Default with
{
    SnapshotUpdateStrategy = SnapshotUpdateStrategy.Disallow,
};

Snapshot.Validate(value, SnapshotType.Default, settings);

Which one to use

  • Disallow keeps the test run free of side effects: approve the actual files with the approval tool when you are happy with them. It is the right default for shared machines and automation.
  • Overwrite suits a change that updates many snapshots on purpose: run the tests once, review the diff in your version control tool, and run them again to check they pass. The first run fails, so you cannot miss that snapshots changed.
  • OverwriteWithoutFailure records new snapshots in a single run. Use it for a run you review afterwards, never as a permanent setting: it hides every regression.
  • MergeTool and MergeToolSync suit an interactive session in an IDE: each change opens in your diff tool, and you keep the parts you want. See Merge tools.

Rewrite every snapshot

ForceUpdateSnapshots updates the verified files even when they match, for instance after changing the serializer settings to reformat every snapshot. Combine it with a strategy that writes files:

SnapshotSettings.Default.ForceUpdateSnapshots = true; // Rewrite the snapshots, even the ones that match
SnapshotSettings.Default.SnapshotUpdateStrategy = SnapshotUpdateStrategy.OverwriteWithoutFailure;

Custom strategies

SnapshotUpdateStrategy is an abstract class. Derive from it to decide, with CanUpdateSnapshot, whether the verified files of an assertion can be updated, with MustReportError whether the test fails, and with UpdateFile or UpdateFiles how the files are updated.