Add Test.AddConfigReload() autest extension#13075
Add Test.AddConfigReload() autest extension#13075brbzull0 wants to merge 2 commits intoapache:masterfrom
Test.AddConfigReload() autest extension#13075Conversation
Replace the fragile pattern of fire-and-forget `traffic_ctl config reload` followed by sleep/log-grepping with a deterministic helper that uses monitor mode (`-m`) to block until completion and validates reload tasks via the get_reload_config_status JSONRPC endpoint. The extension supports expected exit codes (success/fail/timeout/any), task presence and absence assertions, per-task status validation, custom tokens, and a delay_start parameter for filesystem timestamp sensitivity. Migrate 21 existing tests to use the new helper, removing manual sleep synchronization and When.FileContains polling. Add an optional filename parameter to ConfigReloadTask so sub-tasks carry their associated config file for precise identification in test assertions. Includes RST documentation for the new extension API.
There was a problem hiding this comment.
Pull request overview
Adds a deterministic AuTest helper for traffic_ctl config reload that blocks until completion (monitor mode) and optionally validates reload task structure via the get_reload_config_status JSONRPC endpoint, then migrates existing gold tests to use it. Also extends reload task tracing so sub-tasks can report an associated config filename to enable precise test assertions.
Changes:
- Introduce
Test.AddConfigReload()AuTest extension with exit-code expectations and optional task presence/absence + per-task status validation via JSONRPC. - Extend the reload framework to propagate an optional
filenameinto dependent/sub tasks so JSONRPC status can identify which config file triggered each handler. - Migrate a broad set of gold tests to remove sleep/log-grep synchronization in favor of the new helper; add RST documentation for the new API.
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/gold_tests/autest-site/config_reload.test.ext | New AuTest extension implementing Test.AddConfigReload() with optional JSONRPC task validation. |
| src/mgmt/config/ConfigReloadTrace.cc | Update ConfigReloadTask::add_child() to accept/propagate an optional filename into the task context. |
| include/mgmt/config/ConfigReloadTrace.h | Update add_child() API to accept an optional filename parameter. |
| src/mgmt/config/ConfigContext.cc | Update add_dependent_ctx() to accept/forward an optional filename into child tasks. |
| include/mgmt/config/ConfigContext.h | Update add_dependent_ctx() API to accept an optional filename parameter. |
| src/iocore/net/SSLClientCoordinator.cc | Populate dependent reload tasks with sni.yaml / ssl_multicert.yaml filenames for traceability. |
| tests/gold_tests/traffic_ctl/remap_inc/remap_inc.test.py | Replace manual reload + wait script with Test.AddConfigReload() task assertion. |
| tests/gold_tests/tls/tls_verify4.test.py | Replace reload + sleep with Test.AddConfigReload(). |
| tests/gold_tests/tls/tls_tunnel.test.py | Replace reload + log polling with Test.AddConfigReload() and remove log-based wait. |
| tests/gold_tests/tls/tls_sni_yaml_reload.test.py | Switch to Test.AddConfigReload() for expected-fail reload path; disable default log checks. |
| tests/gold_tests/tls/tls_client_cert_plugin.test.py | Replace reload + log polling / delays with Test.AddConfigReload(). |
| tests/gold_tests/tls/tls_client_cert_override_plugin.test.py | Replace reload + log polling with Test.AddConfigReload(). |
| tests/gold_tests/tls/tls_client_cert.test.py | Replace reload + log polling / delays with Test.AddConfigReload(). |
| tests/gold_tests/tls/tls_check_cert_selection_reload.test.py | Replace reload + log polling with Test.AddConfigReload(). |
| tests/gold_tests/tls/ssl_multicert_loader.test.py | Switch to Test.AddConfigReload() for expected-fail reload path; disable default log checks. |
| tests/gold_tests/tls/ssl_key_dialog.test.py | Replace multi-step “await reload” pattern with a single Test.AddConfigReload(). |
| tests/gold_tests/remap_yaml/remap_reload_yaml.test.py | Replace reload + polling with Test.AddConfigReload() (including delayed start). |
| tests/gold_tests/remap_yaml/remap_acl_yaml.test.py | Replace custom reload-and-wait scaffolding with Test.AddConfigReload(). |
| tests/gold_tests/remap/remap_reload.test.py | Replace reload + polling with Test.AddConfigReload() (including delayed start). |
| tests/gold_tests/remap/remap_acl.test.py | Replace custom reload-and-wait scaffolding with Test.AddConfigReload(). |
| tests/gold_tests/pluginTest/regex_revalidate/regex_revalidate_miss.test.py | Split “touch + reload” into deterministic steps using Test.AddConfigReload(). |
| tests/gold_tests/parent_config/parent_config_reload.test.py | Replace reload + long sleep/log polling with Test.AddConfigReload(). |
| tests/gold_tests/logging/log_retention.test.py | Replace reload command with Test.AddConfigReload(). |
| tests/gold_tests/ip_allow/ip_category.test.py | Replace reload + sleep/log polling with Test.AddConfigReload(). |
| tests/gold_tests/ip_allow/ip_allow_reload_triggered.test.py | Replace multiple reload + sleep/log-grep sequences with Test.AddConfigReload() (including absent-task assertion). |
| tests/gold_tests/dns/splitdns_reload.test.py | Replace reload + sleep/log polling with Test.AddConfigReload(). |
| tests/gold_tests/cache/cache_config_reload.test.py | Replace reload invocations with Test.AddConfigReload() using explicit tokens and expected tasks. |
| doc/developer-guide/testing/index.en.rst | Add the new config reload extension documentation to the testing index. |
| doc/developer-guide/testing/config-reload-ext.en.rst | New RST documentation describing Test.AddConfigReload() and its parameters. |
| doc/developer-guide/config-reload-framework.en.rst | Update guidance to recommend Test.AddConfigReload() for end-to-end reload testing. |
| cmd = f"traffic_ctl config reload -m -t {token}" | ||
| cmd += f" -w {initial_wait} -r {refresh_int}" | ||
|
|
||
| if timeout is not None: | ||
| cmd += f" -T {timeout}" | ||
| if force: | ||
| cmd += " --force" | ||
| if data is not None: | ||
| if data.startswith("@"): | ||
| cmd += f" --data {data}" | ||
| else: | ||
| cmd += f" --data {_shell_quote(data)}" |
There was a problem hiding this comment.
The token and data (when passed as @path) are interpolated into a shell command without quoting. If they contain spaces or shell metacharacters the command can break or be interpreted unexpectedly. Shell-quote token and any --data argument (including @... paths), or build the command using an argv-style API if available.
There was a problem hiding this comment.
Good catch. While token is auto-generated (no metacharacters) and data paths come from test code, it's cheap to make this defensive. Fixed — both token and data (including @path form) are now shell-quoted via _shell_quote().
|
[approve ci] |
1 similar comment
|
[approve ci] |
Replace the fragile pattern of fire-and-forget
traffic_ctl config reloadfollowed by sleep/log-grepping with a deterministic helper that uses monitor mode (-m) to block until completion and validates reload tasks via the get_reload_config_status JSONRPC endpoint.The extension supports expected exit codes (success/fail/timeout/any), task presence and absence assertions, per-task status validation, custom tokens, and a delay_start parameter for filesystem timestamp sensitivity.
Migrate 21 existing tests to use the new helper, removing manual sleep synchronization and When.FileContains polling. Add an optional filename parameter to ConfigReloadTask so sub-tasks carry their associated config file for precise identification in test assertions.
Tiny change to the reload core:
Reload sub-tasks previously only carried a description (e.g."SNIConfig"), making it impossible to assert which config file triggereda task. The C++ change adds an optional
filenameparameter toConfigReloadTask::add_child()andConfigContext::add_dependent_ctx()so the JSONRPC status endpoint returns the actual file path (e.g.sni.yaml,remap.yaml) for each sub-task. This allows tests to validate that the right config handlers ran -- or didn't run.Includes RST documentation for the new extension API.
Fixes #12965