Skip to content

APF v2.0.2#52

Open
rfxn wants to merge 81 commits intomasterfrom
2.0.2
Open

APF v2.0.2#52
rfxn wants to merge 81 commits intomasterfrom
2.0.2

Conversation

@rfxn
Copy link
Copy Markdown
Owner

@rfxn rfxn commented Apr 6, 2026

v2.0.2 Release

APF 2.0.2 is a major feature and hardening release: 159 files changed, +31k/-6k lines across 45 commits.

New Features

  • GeoIP country blocking -- ipset-based country filtering with cc_deny/cc_allow rules, ISO 3166-1 codes, continent shorthand (@eu, @as, etc.), tiered data sources, atomic ipset swap, dual-stack IPv4/IPv6, advanced per-port/protocol syntax, CLI management (--cc, --cc-update)
  • Connection tracking limit (CT_LIMIT) -- per-IP connection limit via periodic conntrack scanning with configurable thresholds, port/state filters, CIDR exemptions, VNET overrides, temp-deny blocking with permanent escalation
  • SYN flood protection -- rate-limited inbound TCP SYN via SYNFLOOD chain with configurable rate/burst, LOG + DROP, dual-stack
  • SMTP outbound blocking -- restrict outbound SMTP to whitelisted users/groups
  • Temporary trust with per-entry TTL -- apf -ta/-td with auto cron expiry, --templ/--tempf management, repeat-offender escalation to permanent block
  • Structured event logging (elog_lib) -- JSONL audit trail at /var/log/apf/audit.log with severity filtering and 23 event types
  • CLI overhaul -- subcommand architecture (trust, cc, config, status, gre, ipset, ct), search (-g), --rules, --info, --lookup, --validate, CSF compatibility aliases
  • Advanced trust syntax -- proto:flow:port:ip format in -a/-d/-ta/-td/-u for protocol/direction/port-scoped rules
  • FQDN pre-resolution -- hostname resolution before iptables loading, multiple A/AAAA records, refresh (apf -e), --lookup reverse resolution
  • Config validation -- startup checks for interfaces, rate/burst formats, connlimit syntax, RAB parameters, logging, sysctl
  • Auto-detection -- USE_IPSET, IPT_LOCK_SUPPORT, DOCKER_COMPAT resolved at startup from system capabilities
  • RPM and DEB packaging -- FHS-compliant layout with backward-compatible symlink farm, Docker build infrastructure, install verification tests
  • Custom hook scripts, silent IP blocking, bash tab completion, per-port connlimit, ipset 7-field format, ban expiry markers
  • Improved install/uninstall -- auto-detect interface and conflicting services, structured output, full artifact cleanup

Bug Fixes

  • Trust validation: anchored grep/sed patterns preventing IP substring collisions, control character rejection, glob expansion protection
  • Trust removal: spec-based iptables rule deletion covering all 4 chains (TALLOW, TDENY, TGALLOW, TGDENY)
  • Fast load: atomic snapshot writes, validation before restore, backend marker tracking for nft/legacy compat
  • Mutex locking added to all trust CLI handlers (closes race with cron --temp-expire)
  • Fixed lgate_mac() using undefined variable, expirebans() $ip variable shadowing, sysctl wrong proc paths, refresh() bare IPv6 handling
  • VNET: reset leaked state between iterations, strip Docker @ifn suffix
  • GeoIP: IPv4-as-FQDN false positive, IFS leak corrupting LOG rules, temp CC expiry destroying permanent entries
  • Port deduplication across all filtering loops
  • Portability: coreutils resolution for pre-usr-merge distros (CentOS 6, Ubuntu 12.04)
  • pkg_lib 1.0.7 to 1.0.8: process substitution fix for bash 4.2 POSIX mode

Changes

  • Vendored libs: elog_lib 1.0.5, pkg_lib 1.0.8, geoip_lib 1.0.5, batsman 1.4.2
  • File-based flock mutex replacing FD-based exec/flock
  • ESTABLISHED,RELATED state tracking moved earlier in INPUT chain
  • Single-pass trust loading with iptables-restore --noflush (O(n) to O(1) lock acquisitions)
  • Consolidated 3 cron files into single cron.d.apf
  • Removed vestigial kernel 2.4 ipchains code and dead binary discovery

Test Coverage

  • 716 core tests + 214 UAT scenarios across 23 files
  • 8-OS CI matrix: Debian 12, CentOS 7, Rocky 8/9, Ubuntu 20.04/24.04, CentOS 6, Ubuntu 12.04
  • Full audit: 62 findings (0 critical, 0 major), all resolved

Release Checklist

  • Full distro matrix green (8/8 OS targets)
  • GHA CI green (lint + all matrix targets)
  • shellcheck -S warning passes on all shell files
  • RPM/DEB package build and install verified
  • Legacy compat symlinks verified on FHS package install
  • Tag v2.0.2 after merge
  • GitHub release with CHANGELOG.RELEASE body

rfxn added 30 commits March 30, 2026 19:08
rfxn added 5 commits April 5, 2026 17:48
[Change] PARALLEL_JOBS: nproc (default) → 3 for test-all-parallel
  — 3 OS × 6 groups = 18 privileged containers max
  — prevents iptables kernel state exhaustion and group timeouts
[Change] CHANGELOG: drop internal-only batsman version bump entry,
  move misplaced [Fix] pkg_lib entry from Changes to Bug Fixes section
[Change] tests: merge 12 split-assertion tests into their parent tests
  — same-command pairs checking different output sections consolidated
  into single tests with all assertions (zero coverage lost)
…ed stubs

[Change] Remove vestigial kernel 2.4 ipchains removal block from modinit()
  — no supported OS runs kernel 2.4 (CentOS 6 minimum is 2.6.32)
[Change] Remove dead binary discovery variables from internals.conf:
  UNAME (zero consumers), LSM/RMM (only used by kernel 2.4 block),
  KREL (only used by kernel 2.4 block), global TIME (shadowed by local
  redeclaration in all 4 consumer functions); saves 5 subshell forks
  at startup
[Change] Remove unimplemented trust flush --deny/--allow stubs and update
  error message to only reference --temp (the only implemented option)
[Change] tests/infra: align submodule pointer with CI workflow pin
Copilot AI review requested due to automatic review settings April 6, 2026 06:45
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Release v2.0.2 updates APF with major new security features (GeoIP country filtering, CT_LIMIT, SYN flood controls, SMTP blocking), a modularized CLI/library layout, and hardened operational behavior (auto-detection, atomic snapshotting, structured logging).

Changes:

  • Modularize internals into apf_*.sh libraries and overhaul the CLI into subcommands with improved help/search/compat aliases.
  • Add new firewall features and configuration: CT_LIMIT scanning, GeoIP CC filtering, per-port connlimit, SMTP outbound blocking, SYN flood protection, hooks, and structured audit logging.
  • Consolidate cron jobs and update packaging/CI metadata (submodules, workflows, ignore rules) for the 2.0.2 release.

Reviewed changes

Copilot reviewed 47 out of 159 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
files/internals/internals.conf Bump version, add runtime auto-detection (ipset/docker/iptables lock), state-match selection, and elog defaults.
files/internals/gretunnel.sh GRE validation/robustness tweaks; adds new apf gre dispatch/help.
files/internals/cports.common Refactor port/ICMP filtering loops; add connlimit and SMTP outbound blocking chain.
files/internals/apf_validate.sh New centralized config/input validation helpers (TTL parsing, host/cc validators, config checks).
files/internals/apf_ipt.sh New iptables helpers, backend detection, atomic snapshot saving, module loading helper.
files/internals/apf_ipset.sh New ipset lifecycle (load/update/flush), 7-field migration, atomic swap population.
files/internals/apf_dlist.sh New unified download + remote deny-list parsing/loading utilities.
files/internals/apf_ctlimit.sh New CT_LIMIT conntrack scanner + CLI dispatch/status.
files/internals/apf_cli.sh New CLI UX helpers (help, search, did-you-mean, dispatch scaffolding).
files/internals/apf.lib.sh New sourcing hub for internals + shared micro-utilities and temp cleanup.
files/hook_pre.sh New pre-load hook stub for custom iptables logic.
files/hook_post.sh New post-load hook stub for custom iptables logic.
files/gre.rules Version bump header for GRE rules file.
files/firewall Removes legacy monolithic firewall script (migrated into libs/core).
files/extras/importconf Rewrite upgrade import/merge logic via pkg_lib; add migrations and restore additional rule files.
files/extras/get_ports Improve standalone behavior + output formatting via pkg_* helpers.
files/conf.apf Version bump + extensive new config options (auto-detect toggles, CT_LIMIT, CC filtering, SMTP block, SYNFLOOD, connlimit, elog audit path).
files/cc_deny.rules New country deny rules template/docs.
files/cc_allow.rules New country allowlist rules template/docs with strict allowlist warning.
files/apf CLI overhaul: elog init, flock gate, subcommands/aliases, improved help/errors, version bump.
files/VERSION Bump to 2.0.2.
cron.daily Remove old daily restart cron file.
cron.d.apf_ipset Remove old ipset cron entry (consolidated).
cron.d.apf New unified cron entry (restart, ipset update, temp expiry).
apf.init Version bump + quoting fixes for SysV init wrapper.
apf.bash-completion New bash completion for top-level flags and subcommands.
SECURITY.md Add security policy and disclosure process.
CONTRIBUTING.md Add contribution guidelines and dev/test expectations.
CHANGELOG.RELEASE Replace with 2.0.2 release notes (condensed).
CHANGELOG Add 2.0.2 entry at top (retains 2.0.1 below).
.gitmodules Add batsman test infra submodule.
.github/workflows/smoke-test.yml CI workflow refactor: reuse batsman workflow + add UAT job.
.gitattributes Export-ignore dev/CI/support artifacts.
.dockerignore Reduce Docker context for builds.
.ca.def Remove legacy config template generator.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 959cd462df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@rfxn rfxn changed the title Release v2.0.2 APF v2.0.2 Apr 6, 2026
rfxn added 6 commits April 6, 2026 03:05
[Fix] CT_LIMIT: replace valid_ip_cidr() with valid_host() at conntrack
      offender validation gate; IPv6 addresses from conntrack were silently
      dropped before reaching the blocking path
[Fix] CT_LIMIT: add IPv6 CIDR parsing and prefix matching to awk exempt
      list; IPv6 CIDRs from allow_hosts.rules and CT_SKIP were silently
      ignored, allowing exempt IPv6 addresses to be blocked
[Fix] bash-completion: apf cc <TAB> now shows both verbs (info, lookup,
      update) and country codes; _apf_complete_cc() was overwriting the
      verb list set on the previous line
[New] test: completion functional test verifies COMPREPLY contains both
      verbs and CC entries for position-0 cc group
[Fix] mutex_lock: detect and remove empty lock files left by the flock
      wrapper path; noclobber fallback was timing out after ENTER_LOCK_TIMEOUT
      because the empty-file case skipped the stale PID check
[Fix] gretunnel: default ka_ret to 3 when GRE_KEEPALIVE contains only an
      interval value; single-field input left retries empty, causing silent
      failure of ip tunnel change keepalive command
[Change] CHANGELOG: add 4 fix entries for CT_LIMIT IPv6, GRE keepalive,
         bash-completion cc verbs, and mutex empty lock file
[Change] RPM: require iptables-nft explicitly on RHEL 10+ (sole iptables
         provider); recommend kernel-modules-extra for netfilter compat
         modules needed by RAB, connlimit, and legacy iptables match targets
rfxn added 14 commits April 6, 2026 10:01
[Fix] pkg: Dockerfile.rpm-el7 missing --define "rhel 7" and "el7 1";
      spec %if conditionals for init.d were not evaluating correctly
…gration

[New] RPM/DEB: detect default network interface via ip route on fresh
      install; sets IFACE_UNTRUSTED in conf.apf when system default
      differs from eth0
[Fix] RPM/DEB: remove iptables snapshots, change-detection markers,
      runtime cron templates, and lock files during package removal;
      orphan files were preventing /etc/apf/internals/ directory cleanup
[Fix] RPM/DEB: remove pre-decomposition files (firewall, functions.apf,
      geoip.apf, ctlimit.apf) and legacy cron entries during migration
      from install.sh-based installations
[Fix] DEB: add util-linux to Depends for flock (consistent with RPM)
[Fix] RPM: pass explicit BK_LAST to importconf instead of relying on
      fallback symlink resolution
[Change] RPM/DEB: document intentional omissions (service enable,
         VNET generation) with inline comments per packaging guidelines
[Fix] RPM/DEB: add inline comment on ip route 2>/dev/null per governance
[Fix] DEB: pass explicit BK_LAST to importconf for RPM/DEB symmetry
[Fix] RPM/DEB: add .apf6.restore* (IPv6 iptables snapshot) and
      .block_history* (CT_LIMIT block tracking) to package removal
      cleanup; both were orphaned after uninstall
[Fix] RPM/DEB: add baseline files (.apf.input.baseline, .apf6.*),
      fast-load timestamp (.last.full), address caches (.localaddrs,
      .localaddrs6, .trusts.md5), CT scan timestamp (.ct_last_scan),
      ipset timestamps, and GRE tunnel tracking to removal cleanup;
      change .apf-* removal from rm -f to rm -rf for geoip temp dirs
…ation

[Fix] RPM/DEB: wipe legacy install path after backup during install.sh
      migration; cpio cannot replace /etc/apf/doc and /etc/apf/extras
      directories with symlinks, causing rpm -ivh to fail with "rename
      failed - Is a directory"; conffiles were also triggering .rpmnew
      on first install over an existing tree, causing importconf to merge
      old config into old config instead of into the new template
[Fix] RPM: %pre aborts install on backup failure instead of silently
      continuing to wipe legacy_path; matches DEB preinst set -e
      behavior; prevents data loss if cp -a fails
[Fix] importconf: restore internals/internals.conf via pkg_config_merge
      from backup; the file is a %config(noreplace) conffile in both
      RPM and DEB but was not in the restoration list, so user
      customizations (path overrides, timeouts, IPT_FLAGS, MD5_FILES)
      were silently lost during install.sh migration after %pre wiped
      the legacy tree
…ile parity

[Fix] RPM: move install.sh migration from %pre to %pretrans. RPM locks
      the %config(noreplace) extract-vs-.rpmnew decision before %pre runs,
      so wiping the legacy /etc/apf in %pre still produced .rpmnew files
      for any user-modified conffiles AND left no /etc/apf/conf.apf at
      all (only conf.apf.rpmnew), which then broke importconf in %post
      with "cp: cannot stat '/etc/apf/conf.apf'" and "pkg_config_merge:
      new config not found". %pretrans runs before the conffile scan, so
      RPM sees an empty /etc/apf and extracts cleanly with no .rpmnew.
      Affects fresh rpm -ivh over an existing install.sh-based install
      with locally-edited conf.apf or *_hosts.rules.
[Fix] RPM/DEB preinst: stop SysV init loop now breaks after the first
      hit. /etc/init.d is a symlink to /etc/rc.d/init.d on RHEL, so the
      previous loop invoked the same init script twice and printed
      "Stopping APF: done" twice.
[Fix] importconf: drop pkg_config_merge call for internals.conf. The
      file contains conditional assignments (if/else STATE_MATCH=,
      NET=, IPT_FLAGS=) whose branches collapse to a single value
      under merge, and shell conditionals like `[ "$X" = "1" ]` /
      `[[ "$Y" == "auto" ]]` were corrupted into `[ "$X"= "1" ]` /
      `[[ "$Y"== "auto" ]]` by the pre-1.0.9 merge function. New
      template is correct on its own — most contents are
      auto-detected at runtime. Failure mode: every smoke-test OS
      target reported "internals.conf: line N: conditional binary
      operator expected" plus elog_init / mutex_unlock command not
      found errors after a re-install over an existing /opt/apf
      (introduced 07c067a, broke run 24064840228 across all 8 OS
      matrix entries on PR #52).
[Fix] pkg_lib v1.0.9: pkg_config_merge anchors variable detection to
      a real shell assignment regex
      (^[[:space:]]*[A-Za-z_][A-Za-z0-9_]*=). Pre-1.0.9 the merge
      treated any line containing `=` as VAR=value, stripping spaces
      from conditional expressions and producing syntactically broken
      bash. Conditional and other shell-code lines now pass through
      verbatim. Vendored copy synced byte-identical with canonical;
      regression test added in canonical
      pkg_lib/tests/09-config.bats covering bash conditional
      passthrough plus a `bash -n` parse check on the merged output.
[Fix] apf --cc <CC> and apf cc info <CC> emitted "geoip_cc_known: command
      not found" followed by "Unknown country code: <CC>" for every input
      on production. Root cause: the function was added to the APF
      vendored copy in commit 2af788a (Mar 22) without canonical-first
      promotion, then silently deleted by the Apr 5 canonical->vendored
      sync in commit 5e64a5e. UAT tests only exercised the unknown-CC
      path (ZZ) which coincidentally matched the command-not-found
      fallback output, so automated gates missed the regression.
[Change] files/internals/geoip_lib.sh: sync from canonical geoip_lib
         1.0.5 -> 1.0.6 which restores geoip_cc_known() at the source.
[New] tests/uat/20-geoip-cli.bats: UAT-GC06b valid-CC detail test
      (apf --cc US) — would have caught the regression on day one.
      Also adds refute_output "command not found" regression guard to
      the existing UAT-GC06 ZZ unknown-CC test.
…1.0.7

[Change] elog_lib 1.0.5 → 1.0.6: 44-line ELOG_* config catalogue at file
        header replaced with README.md § 3 cross-reference; 26 banner
        separator lines removed. SIEM/CEF/GELF/ECS/API contract docs and
        the 8 load-bearing function/section headers preserved.
[Change] pkg_lib 1.0.9 → 1.0.10: 206 signature-restatement lines removed
        from 42 multi-line function-header blocks. Load-bearing prose
        preserved (pkg_config_merge invariants, AWK/eval injection
        rationale, GNU/BSD stat splits, FreeBSD guards, CentOS 6 usr-merge,
        bash 4.1 indirect-expansion notes, detection-chain documentation).
[Change] geoip_lib 1.0.6 → 1.0.7: 56 banner separator lines and 45
        signature-restatement header lines removed. TLS posture (strict
        default + GEOIP_TLS_INSECURE escape hatch), curl/wget preference
        chain, mawk-compat, .last_update symlink-safety, and lock contract
        all preserved.

Zero functional change in any of the three libraries — all bumps are
comment-discipline cascade work from the canonical RDF core profile.

Verification:
- Pre-flight function-list diff (per APF Shared-Library Sync Preflight)
  shows no vendored-only functions in elog/pkg/geoip — clean canonical
  superset. No regression risk equivalent to the geoip_cc_known() incident.
- Byte-identical to canonical: elog_lib 7a60165 (v1.0.6),
  pkg_lib cdd0814 (v1.0.10), geoip_lib cc9559f (v1.0.7).
- bash -n + shellcheck -S error clean on all three vendored copies.
- Tests on anvil: 720/720 passed Debian 12 (204s), 720/720 Rocky 9 (242s).
[Change] Delete 27 '# Args:' / '# Arguments:' / '# Params:' signature-
        restatement blocks and 9 '# Usage:' function-signature lines across
        13 APF-owned shell files per RDF comment-discipline rule.
[Change] Prune exact-match boilerplate '# Returns 0 on success, 1 on failure.'
        (2 lines) and near-boilerplate '# Returns' lines (5 per engineer
        judgment). Collapse marginal algorithm-restatement header lines
        in snapshot_save() and _ct_build_exempt(). Zero functional change.
[New] Add comment-discipline grep gate to .rdf/governance/verification.md
      § "Comment Discipline Greps" — four patterns: signature-restatement
      (Args|Arguments|Parameters|Params|Usage), banner separators (#[#=_-]{9,}),
      tombstone markers (removed|was:|legacy|deprecated), and boilerplate
      '# Returns 0 on success, 1 on failure.'. Any hit is a reviewer finding.
[New] Add three comment-discipline greps to workspace parent CLAUDE.md
      § "Verification Before Commit" for cross-project enforcement.
[Change] Merge 3 dated sub-headers (Apr 11, Apr 08, Mar 08) into single
version block with unified New Features / Bug Fixes / Changes sections.
Drop 4 internal-only entries (comment discipline, governance gate, UAT
test, superseded lib sync). Fold 2 sub-step entries into parent fixes.
Merge 4 groups: importconf 3→1, RPM/DEB packaging 7→2, GeoIP 2→1,
install.sh 2→1. Move RPM/DEB auto-detect interface from Bug Fixes to
New Features (tag mismatch). Trim verbose entries. CHANGELOG and
CHANGELOG.RELEASE in sync.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants