ABSTRACT
This paper presents a practical accuracy rating system for evaluating discrete probability predictions. The approach assigns credit based on the probability given to the correct outcome, with an optional time-weighting feature that rewards earlier, riskier predictions more than late-breaking ones. The model uses Bayesian methods to estimate each predictor’s skill level and determine the probability that they are doing better than random guessing. A baseline comparison against a discrete uniform probability distribution (DUPD) enables objective performance evaluation. This framework is flexible enough to handle real-time updates and live predictions, merging them with pre-event forecasts into a single, interpretable metric. The model is demonstrated by evaluating Super Bowl LIX predictions from five leading sportsbooks and the prediction market Polymarket. The result is a robust, scalable system for benchmarking predictive performance.