Info
When a PR is submitted with the evergreen github PR bot (commenting "evergreen merge"), the commit queue run will use the evergreen ymls at the time of submission, rather than the evergreen ymls at HEAD.
Example: https://parsley.mongodb.com/evergreen/mongodb_mongo_master_amazon_linux2_arm64_dynamic_compile_clang_tidy_4_patch_9443e40f3ca6016a3068e69ad46c897aa2483976_66145a23ceacf90007238308_24_04_08_21_06_19/0/task?bookmarks=0%2C868&selectedLineRange=L472&shareLine=472
This can result in two critical issues:
1. Letting commits through that would effectively break the commit queue for everyone (allowing an incompatibility with HEAD)
2. Breaking PR submissions for anyone that schedules a merge right before someone else schedules a change that breaks cross-version compatibility between the evergreen ymls and the underlying repo (ex. adding a reference to a newly committed script file to an existing evergreen yml function)
These issues will go away when we've switched to the github merge queue, but for until then may cause serious problems.
I'm not sure if we intend to solve this problem or wait until the merge queue, but we should have a ticket to reference the issue.