Why does the price per person go down when there are more of us?

This only happens on flights offered with an equal split. The flight's costs are fixed: they don't change with the number of people on board. The more of you there are to share them, the less each person pays. The pilot always counts among the occupants who split the costs.

The principle, plainly

The flight's direct costs are divided between all occupants, including the pilot. With two occupants, they're split in two; with three, in three; and so on. Each extra person therefore lowers everyone's share.

With a fixed contribution, the price doesn't change

On a flight with a fixed contribution, the amount per seat is set in advance by the pilot. It stays the same regardless of the number of bookings. So this mode brings no reduction when there are more of you.