Rebuild our Prow Build Clusters to support dualstack networking

[upodroid]

Our Prow build clusters don't support IPv6, and IPv6 has been GA in Kubernetes for several years:

The GKE cluster running the prow control plane has IPv6 enabled already.

Clusters in scope:

k8s-infra-prow-build
k8s-infra-prow-build-trusted
eks-prow-build-cluster
k8s-infra-aks-admin (This cluster never worked)

Runbook:

1. Fix all the following outstanding issues:
   1. KES: Migrate to SecretStore from ExternalSecret #3123
   2. eks-prow-build-cluster: Replace FluxCD with ArgoCD #6442
2. Provision new Infrastructure
   1. New dual-stack VPCs for gke prow build clusters
   2. Modify the existing AWS VPC to support IPv6
   3. Deploy new GKE clusters like this: https://github.com/kubernetes/k8s.io/blob/main/infra/gcp/terraform/k8s-infra-prow/gke.tf
   4. Delete the old clusters
   5. Deploy the new clusters
3. Post cluster provisioning
   1. Update the endpoints of the clusters in ArgoCD, https://github.com/kubernetes/k8s.io/blob/main/kubernetes/gke-utility/argocd/clusters.yaml
   2. Update the endpoints of the clusters in Prow by modifying the kubeconfig secrets in GCP Secret Manager

We expect this operation to take 3 hours and ideally be executed on the weekend before the start of v1.36 release cycle.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[xmudrii]

/remove-lifecycle stale