[spark] Expose Paimon metrics via JMX for Prometheus scraping

[junmuz]

Purpose

Bridge Paimon commit metrics into Spark's MetricsSystem using a Codahale MetricRegistry so they are accessible as JMX MBeans. This enables jmx_prometheus_javaagent to scrape Paimon metrics (commit duration, files added/deleted, records appended, partitions written, etc.).

• Add SparkMetricGroup extending MetricGroupImpl to dual-register metrics in both Paimon's internal registry and a shared Codahale registry
• Add PaimonMetricsSource singleton with its own JmxReporter to register MBeans immediately (bypasses Spark MetricsSystem snapshot limitation)
• Wire SparkMetricRegistry to use SparkMetricGroup
• Add withMetricRegistry() to V1 commit path in PaimonSparkWriter

Tests

• Unit tests for SparkMetricGroup: gauge/counter/histogram registration, lazy value updates, close cleanup, name sanitization, collision handling, backward compatibility with getMetrics()
  • Integration tests verifying commit metrics appear in the Codahale registry after writes and update correctly on subsequent commits