Improve database response time for listing user activity

Closes #40525

See merge request gitlab-org/gitlab-ce!17454