Improved ref switcher dropdown performance

Closes #18202

See merge request !7239