Rocky has the same packages as RHEL. So if RHEL did it that way, then you need to ask them why. Rocky is a 1:1 rebuild of RHEL so the package versions are the same as RHEL.
Note that platform-python in RHEL 8 is 3.6 and in RHEL 9 it is 3.9.
The Ansible depends on Python. On RHEL 9 it does use the 3.9.
On RHEL 8 the Ansible depends on application stream python3.12 – not the platform-python.
Could use of newer Ansible be connected to use of Python?
The RHEL 8 did not release with ansible-core. That did happen in 8.6. See Updates to using Ansible in RHEL 8.6 and 9.0
I’d guess that more work was necessary in RHEL 8 – that ansible-core could not use python3.6 – and hence it did diverge.