"Points" were originally a way to talk about effort in a dysfunctional (from an agile-planning perspective) environment that focused on time-based rather than priority-based planning.