Skip to main content

Reporting and evidence

SHAFT records test actions as structured Allure steps and attaches the evidence needed to understand failures.

Generated element assertions keep the Allure step text concise: the locator is reported once as a step parameter, while internal element reads used to evaluate the assertion are kept in the debug execution log instead of repeated child steps.

Text-entry and dropdown element actions keep the Allure timeline readable by showing the value being typed or selected in the step title. Long or multiline values are capped in the title. The step metadata keeps the normalized locator, including Smart Locator labels, and includes the resolved element name only when the engine captures one. Secure typing stays masked.

Use reporting configuration and custom report messages for detailed controls.

src/main/resources/properties/custom.properties
screenshotParams_whenToTakeAScreenshot=ValidationPointsOnly
reporting.attachFullLog=true
createAnimatedGif=false

Execution logs​

SHAFT writes the engine execution log through asynchronous Log4j2 appenders so normal test actions do not wait on file I/O. The console shows the concise INFO-level story, while diagnostic entries and engine internals are written at DEBUG level to the log file.

Set reporting.attachFullLog=true when you want the full engine log attached to the Allure report after execution. The attachment is streamed from a temporary deduplicated snapshot so the live target/logs/log4j.log file remains available for retry diagnostics, local investigation, and CI artifact collection.

Failure diagnostics bundle​

Failed and broken tests attach shaft-diagnostics.zip to Allure by default. The ZIP contains diagnostics.json, a sanitized handoff file for humans, SHAFT Doctor, and MCP clients. It includes stable test metadata, failure type, message, stacktrace, root-cause chain, the top project stack frame, bounded logs, referenced artifacts such as videos, GIFs, and trace viewer files, selected runtime/configuration metadata, redaction rules applied, size limits, and suggested Doctor/MCP commands.

The bundle references existing artifacts instead of copying every raw file. Secrets in common headers, cookies, token/password assignments, and sensitive URL query values are masked before the JSON is attached. Disable the bundle with shaft.diagnostics.enabled=false or lower the ZIP entry cap with shaft.diagnostics.maxArtifactMb.

Locator health reports​

Enable locator health reporting when you want a run-level view of slow or flaky web locators without changing test code.

src/main/resources/properties/custom.properties
shaft.locatorHealth.enabled=true
shaft.locatorHealth.warnBelowScore=70
shaft.locatorHealth.attachDashboard=true
shaft.locatorHealth.failBelowScore=-1
slowLocatorThresholdMillis=750

When enabled, SHAFT records lookup counts, unique/no-match/multi-match/stale rates, average and p95 lookup time, polling attempts, timeouts, slow lookups, SHAFT Heal attempts, accepted recoveries, confidence, and selected replacement locators when available. It scores each locator, flags selector smells such as absolute XPath, index-heavy XPath, generated IDs, text-only selectors, and deep CSS chains, then adds plain-language recommendations.

At the end of the run it writes HTML and JSON reports under execution-summary/locator-health/ and attaches the JSON export to Allure. The HTML dashboard is attached when shaft.locatorHealth.attachDashboard=true. Keep shaft.locatorHealth.failBelowScore=-1 while introducing the report; set it to a score threshold only after the suite has a stable baseline. The older locatorHealthReportEnabled=true key remains supported. When the failure trace viewer is enabled, failed-test trace JSON also includes the current locator health snapshot.