Scaling limits for critical queues with diminishing populations