Prevent horizontal scrolling in new sidebar when collapsed

Closes #36608 and #36507

See merge request !13638