Decide what wakes someone
Only what users feel right now: the app down, crash loops, a burst of server errors, payments failing. Everything else is a message or a weekly summary.
Choose the order
One person first, one backup. Five minutes is a reasonable wait before moving on.
Give the alert context
What broke, since when, what changed. An alert that says only "error rate high" costs 20 minutes at 3 a.m.
Set quiet hours for everything else
Non-urgent messages wait for the morning.
Review every page
Weekly: did it need a human? If not, make it quieter. If something was missed, make it louder.
Rotate fairly
With more than two people, rotate who is first each week. Use the on-call schedule maker to draft it.
Pagerless does steps 1 to 5 out of the box; it has no rotation schedules yet, so rotate the first person by hand.