Skip to content

Add Test.AddConfigReload() autest extension#13075

Open
brbzull0 wants to merge 2 commits intoapache:masterfrom
brbzull0:autest_reload_ext
Open

Add Test.AddConfigReload() autest extension#13075
brbzull0 wants to merge 2 commits intoapache:masterfrom
brbzull0:autest_reload_ext

Conversation

@brbzull0
Copy link
Copy Markdown
Contributor

@brbzull0 brbzull0 commented Apr 9, 2026

Replace the fragile pattern of fire-and-forget traffic_ctl config reload followed by sleep/log-grepping with a deterministic helper that uses monitor mode (-m) to block until completion and validates reload tasks via the get_reload_config_status JSONRPC endpoint.

The extension supports expected exit codes (success/fail/timeout/any), task presence and absence assertions, per-task status validation, custom tokens, and a delay_start parameter for filesystem timestamp sensitivity.

Migrate 21 existing tests to use the new helper, removing manual sleep synchronization and When.FileContains polling. Add an optional filename parameter to ConfigReloadTask so sub-tasks carry their associated config file for precise identification in test assertions.

Tiny change to the reload core:

Reload sub-tasks previously only carried a description (e.g."SNIConfig"), making it impossible to assert which config file triggereda task. The C++ change adds an optional filename parameter toConfigReloadTask::add_child() and ConfigContext::add_dependent_ctx()so the JSONRPC status endpoint returns the actual file path (e.g.sni.yaml, remap.yaml) for each sub-task. This allows tests to validate that the right config handlers ran -- or didn't run.

Includes RST documentation for the new extension API.

Fixes #12965

Replace the fragile pattern of fire-and-forget `traffic_ctl config
reload` followed by sleep/log-grepping with a deterministic helper
that uses monitor mode (`-m`) to block until completion and validates
reload tasks via the get_reload_config_status JSONRPC endpoint.

The extension supports expected exit codes (success/fail/timeout/any),
task presence and absence assertions, per-task status validation,
custom tokens, and a delay_start parameter for filesystem timestamp
sensitivity.

Migrate 21 existing tests to use the new helper, removing manual
sleep synchronization and When.FileContains polling. Add an optional
filename parameter to ConfigReloadTask so sub-tasks carry their
associated config file for precise identification in test assertions.

Includes RST documentation for the new extension API.
@brbzull0 brbzull0 added this to the 11.0.0 milestone Apr 9, 2026
@brbzull0 brbzull0 self-assigned this Apr 9, 2026
@brbzull0 brbzull0 marked this pull request as ready for review April 9, 2026 18:43
@zwoop zwoop requested a review from Copilot April 15, 2026 22:07
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a deterministic AuTest helper for traffic_ctl config reload that blocks until completion (monitor mode) and optionally validates reload task structure via the get_reload_config_status JSONRPC endpoint, then migrates existing gold tests to use it. Also extends reload task tracing so sub-tasks can report an associated config filename to enable precise test assertions.

Changes:

  • Introduce Test.AddConfigReload() AuTest extension with exit-code expectations and optional task presence/absence + per-task status validation via JSONRPC.
  • Extend the reload framework to propagate an optional filename into dependent/sub tasks so JSONRPC status can identify which config file triggered each handler.
  • Migrate a broad set of gold tests to remove sleep/log-grep synchronization in favor of the new helper; add RST documentation for the new API.

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/gold_tests/autest-site/config_reload.test.ext New AuTest extension implementing Test.AddConfigReload() with optional JSONRPC task validation.
src/mgmt/config/ConfigReloadTrace.cc Update ConfigReloadTask::add_child() to accept/propagate an optional filename into the task context.
include/mgmt/config/ConfigReloadTrace.h Update add_child() API to accept an optional filename parameter.
src/mgmt/config/ConfigContext.cc Update add_dependent_ctx() to accept/forward an optional filename into child tasks.
include/mgmt/config/ConfigContext.h Update add_dependent_ctx() API to accept an optional filename parameter.
src/iocore/net/SSLClientCoordinator.cc Populate dependent reload tasks with sni.yaml / ssl_multicert.yaml filenames for traceability.
tests/gold_tests/traffic_ctl/remap_inc/remap_inc.test.py Replace manual reload + wait script with Test.AddConfigReload() task assertion.
tests/gold_tests/tls/tls_verify4.test.py Replace reload + sleep with Test.AddConfigReload().
tests/gold_tests/tls/tls_tunnel.test.py Replace reload + log polling with Test.AddConfigReload() and remove log-based wait.
tests/gold_tests/tls/tls_sni_yaml_reload.test.py Switch to Test.AddConfigReload() for expected-fail reload path; disable default log checks.
tests/gold_tests/tls/tls_client_cert_plugin.test.py Replace reload + log polling / delays with Test.AddConfigReload().
tests/gold_tests/tls/tls_client_cert_override_plugin.test.py Replace reload + log polling with Test.AddConfigReload().
tests/gold_tests/tls/tls_client_cert.test.py Replace reload + log polling / delays with Test.AddConfigReload().
tests/gold_tests/tls/tls_check_cert_selection_reload.test.py Replace reload + log polling with Test.AddConfigReload().
tests/gold_tests/tls/ssl_multicert_loader.test.py Switch to Test.AddConfigReload() for expected-fail reload path; disable default log checks.
tests/gold_tests/tls/ssl_key_dialog.test.py Replace multi-step “await reload” pattern with a single Test.AddConfigReload().
tests/gold_tests/remap_yaml/remap_reload_yaml.test.py Replace reload + polling with Test.AddConfigReload() (including delayed start).
tests/gold_tests/remap_yaml/remap_acl_yaml.test.py Replace custom reload-and-wait scaffolding with Test.AddConfigReload().
tests/gold_tests/remap/remap_reload.test.py Replace reload + polling with Test.AddConfigReload() (including delayed start).
tests/gold_tests/remap/remap_acl.test.py Replace custom reload-and-wait scaffolding with Test.AddConfigReload().
tests/gold_tests/pluginTest/regex_revalidate/regex_revalidate_miss.test.py Split “touch + reload” into deterministic steps using Test.AddConfigReload().
tests/gold_tests/parent_config/parent_config_reload.test.py Replace reload + long sleep/log polling with Test.AddConfigReload().
tests/gold_tests/logging/log_retention.test.py Replace reload command with Test.AddConfigReload().
tests/gold_tests/ip_allow/ip_category.test.py Replace reload + sleep/log polling with Test.AddConfigReload().
tests/gold_tests/ip_allow/ip_allow_reload_triggered.test.py Replace multiple reload + sleep/log-grep sequences with Test.AddConfigReload() (including absent-task assertion).
tests/gold_tests/dns/splitdns_reload.test.py Replace reload + sleep/log polling with Test.AddConfigReload().
tests/gold_tests/cache/cache_config_reload.test.py Replace reload invocations with Test.AddConfigReload() using explicit tokens and expected tasks.
doc/developer-guide/testing/index.en.rst Add the new config reload extension documentation to the testing index.
doc/developer-guide/testing/config-reload-ext.en.rst New RST documentation describing Test.AddConfigReload() and its parameters.
doc/developer-guide/config-reload-framework.en.rst Update guidance to recommend Test.AddConfigReload() for end-to-end reload testing.

Comment thread tests/gold_tests/autest-site/config_reload.test.ext
Comment on lines +198 to +209
cmd = f"traffic_ctl config reload -m -t {token}"
cmd += f" -w {initial_wait} -r {refresh_int}"

if timeout is not None:
cmd += f" -T {timeout}"
if force:
cmd += " --force"
if data is not None:
if data.startswith("@"):
cmd += f" --data {data}"
else:
cmd += f" --data {_shell_quote(data)}"
Copy link

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The token and data (when passed as @path) are interpolated into a shell command without quoting. If they contain spaces or shell metacharacters the command can break or be interpreted unexpectedly. Shell-quote token and any --data argument (including @... paths), or build the command using an argv-style API if available.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. While token is auto-generated (no metacharacters) and data paths come from test code, it's cheap to make this defensive. Fixed — both token and data (including @path form) are now shell-quoted via _shell_quote().

Comment thread tests/gold_tests/tls/tls_sni_yaml_reload.test.py
Comment thread tests/gold_tests/tls/ssl_multicert_loader.test.py
@brbzull0
Copy link
Copy Markdown
Contributor Author

[approve ci]

1 similar comment
@brbzull0
Copy link
Copy Markdown
Contributor Author

[approve ci]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ATS Reload migration task. Autest reload extension

2 participants