Vendor Performance Evaluation: How to Build a Vendor Scorecard
· 8 min read · Officeagent research
Try it on a real task
Routing slip · Officeagent
Status: Ready
Action requested
Pick a task above and press RUN IT. Officeagent handles it end to end; you approve the send.
Reading page /
☐ Approve · nothing sends without you
In the product you edit the draft right here; the agent learns your correction.
Officeagent drafted, you approved, it executed.
Sample data · Officeagent always waits for your approval before anything is sent
Vendor performance evaluation is the practice of measuring what a supplier actually delivers against what you agreed, on a schedule, using a small set of measures you can get numbers for. A vendor scorecard is that measurement written down: three to five criteria, a weight on each, a score per period, and a trend. The point is not the number. The point is that the review happens at all, with the vendor in the room, before the renewal date rather than after it.
Most vendor scorecards fail the same way. Someone builds a twenty-criterion template in a spreadsheet, fills it in once with real care, and never opens it again, because gathering the data took four hours and nobody has four hours next quarter. The version that survives is smaller than feels rigorous. Here is how to build one that still exists a year from now.
Where evaluation sits in the vendor lifecycle
Performance evaluation belongs to the ongoing monitoring stage of the vendor management process, which is the stage with no deadline attached to it. Onboarding has a first invoice forcing it. Contracts have a signature date. Payments have terms. Monitoring has nothing, so it is the first thing that quietly stops happening when a quarter gets busy, and the last thing anybody notices has stopped.
That is worth naming because it changes the design goal. You are not trying to build the most accurate assessment of a supplier. You are trying to build the lightest one that will still be running in eighteen months. Accuracy you can add later. A habit you cannot retrofit.
What to actually measure
Five categories cover almost every vendor. Pick from them rather than inventing criteria, and use three to five per vendor, not all of them.
| Category | What it answers | Where the number comes from |
|---|---|---|
| Delivery | Did it arrive on time and complete? | PO dates against receipt dates |
| Quality | Was it right the first time? | Rejections, rework, defect or error count |
| Cost and billing | Did invoices match what was agreed? | Invoice exceptions in your AP records |
| Responsiveness | How did they behave when something went wrong? | Ticket or email response times, incident notes |
| Compliance | Is the paperwork current and the risk profile unchanged? | Insurance and certification expiry dates |
Two of these deserve comment. Billing accuracy is the measure people leave out and the one your own records already contain: if you run a three-way match, every invoice exception is a data point about the vendor, collected automatically as a by-product of paying them. You do not have to gather it, only count it.
Responsiveness under failure is the most predictive measure and the hardest to quantify, which is why it usually gets dropped. Every vendor looks competent when nothing has gone wrong. How they behaved during the outage, the missed delivery, or the billing dispute tells you more about the next three years than any on-time percentage will. Score it honestly on a simple scale and write one sentence of evidence next to it while the incident is fresh. That sentence is the part you will actually reread.
Building the scorecard
Keep the mechanics boring. A five-point scale is enough: 1 is failing, 3 is meeting the agreement, 5 is materially better than the agreement. Anything finer invites arguments about whether a vendor is a 74 or a 78, which is a conversation with no useful output.
Weight the criteria to reflect what you are actually buying. A logistics supplier is mostly delivery. A managed IT provider is mostly responsiveness and compliance. A contract manufacturer is mostly quality. If every criterion carries equal weight, the scorecard tells you a vendor is average when what you needed to know is that it is failing at the one thing you pay it for.
Here is a worked example for a mid-tier facilities vendor, using four criteria:
| Criterion | Weight | Q1 | Q2 | Q3 | Note |
|---|---|---|---|---|---|
| On-time completion | 40% | 4 | 4 | 3 | Two jobs slipped in Q3, both flagged in advance |
| Work right first time | 30% | 5 | 4 | 4 | One callback, resolved same week |
| Invoice accuracy | 15% | 3 | 2 | 2 | Recurring line item billed at the old rate |
| Responsiveness | 15% | 5 | 5 | 5 | Answered the burst pipe call on a Sunday |
| Weighted score | 4.3 | 3.9 | 3.4 | Trend down two quarters running |
Notice what the table surfaces that a single overall rating would hide. This vendor is well liked, answers the phone on a Sunday, and is drifting. The billing problem is small in weight and persistent in fact, which usually means nobody has told them. The trend is the finding, not the score.
How often to evaluate
Set the cadence by vendor tier rather than reviewing everyone equally, which is the fastest route to reviewing nobody. Critical vendors that could stop your business or hold your data warrant a quarterly review. Important but replaceable vendors warrant an annual one, timed to land before the renewal decision rather than after it. Routine commodity suppliers need no scheduled review at all, only a check that the insurance and tax paperwork is current.
Whatever the cadence, schedule the review relative to the contract's notice window, not the calendar year. A thorough annual assessment completed three weeks after the auto-renewal window closed is an interesting document with no consequences attached.
Running the review with the vendor, not about the vendor
Send the scorecard in advance. Walk through it together. Ask what is making their side harder, because a surprising amount of poor vendor performance traces back to something on your end: approvals that take two weeks, a site contact who does not answer, specifications that change after work starts.
A vendor that first hears about a problem in a termination notice never had the chance to fix it, and you never got the benefit of them fixing it. Agree on two or three specific changes, write down who owns each, and open the next review with them. A structured agenda keeps this from turning into a general catch-up, and the follow-up note afterward is what makes the agreed actions survive to the next quarter. If those notes tend to die in someone's drafts folder, the format in how to write a meeting recap email is the one that gets read and acted on.
What to do with a bad score
A single weak quarter is information, not a decision. Two consecutive weak quarters after a documented conversation is a decision, and the mistake is waiting for a third out of politeness or inertia.
When a vendor is genuinely underperforming, run the replacement search in parallel rather than in sequence. Give the incumbent a written improvement plan with specific measures and a deadline, and at the same time start sourcing alternatives so you know what the market actually offers before the renewal conversation. Buyers who begin looking only after deciding to leave end up renewing anyway, because switching under time pressure always looks worse than staying. Having a credible alternative is also the only thing that reliably improves an incumbent's pricing.
If you do move on, run the transition as an overlap: the new vendor onboarded and working before the old one goes dark. Our vendor onboarding checklist covers the paperwork and banking setup for the incoming supplier, and the walkthrough of onboarding a new vendor covers the sequence to run it in when you are switching mid-contract.
Four mistakes that make scorecards useless
- Too many criteria. Twenty measures means the data never gets gathered. Three that you can pull from records you already keep will outperform it every time, because they will actually get pulled.
- Scoring from memory. A rating assigned at review time reflects the last three weeks, not the quarter. Note incidents when they happen, in one line, against the vendor record.
- Never sharing it. A scorecard the vendor has not seen is an internal opinion. It changes nothing about their behavior, which was the entire point.
- No consequence attached. If a 4.5 and a 2.5 lead to identical outcomes at renewal, the exercise is theater and everyone involved works that out within two cycles.
Frequently asked questions
What is vendor performance evaluation?
Vendor performance evaluation is the scheduled measurement of what a supplier delivers against what was agreed, using a defined set of criteria such as on-time delivery, quality, billing accuracy, responsiveness, and compliance. It sits in the ongoing monitoring stage of the vendor management lifecycle, and its output feeds the renewal decision, which is where the leverage is.
How do you measure vendor performance?
Pick three to five measures you can source from records you already keep: purchase order dates against receipt dates for delivery, rework and rejections for quality, invoice exceptions from your accounts payable process for billing, and response times or incident notes for responsiveness. Score each on a five-point scale, weight them to reflect what you are buying, and track the trend across periods.
What is a vendor performance scorecard?
A vendor performance scorecard is a short table holding the criteria you evaluate a supplier on, the weight of each, the score for each period, and a note of the evidence. It is reviewed on a cadence set by the vendor's tier and shared with the vendor. A shared spreadsheet with one row per criterion and one column per quarter is sufficient for most companies.
What are vendor KPIs?
The common ones are on-time delivery rate, order accuracy or defect rate, first-time-right percentage, invoice exception rate, average response and resolution time, and compliance document currency. Choose the two or three that reflect what you actually pay this vendor for. A logistics supplier is judged on delivery; a managed service provider on responsiveness and uptime.
How often should you evaluate vendors?
By tier. Critical vendors that could halt operations or hold your data warrant quarterly reviews. Important but replaceable vendors warrant an annual review, scheduled to complete before the contract's notice window opens. Routine commodity suppliers need only a periodic check that insurance and tax paperwork is current, not a performance review.
What should be included in a vendor performance report?
The criteria and their weights, the score for the current period, the trend across at least two prior periods, one line of evidence per criterion, any incidents with what happened and how the vendor responded, the actions agreed at the last review and whether they were completed, and a recommendation for the upcoming renewal decision. Keep it to one page.
What is the difference between vendor evaluation and vendor due diligence?
Due diligence happens before you sign and asks whether this supplier is capable, legitimate, and safe to work with. Performance evaluation happens after you sign and asks whether they are delivering what they promised. Diligence is a gate; evaluation is a loop. Both feed the same renewal decision, which is why the diligence notes are worth keeping.