Write the criteria before the pilot starts
The scorecard has four parts: gates, scored criteria, a task set and an approval record. Fill in the thresholds and weights before licenses go out, run a baseline window, license the pilot group, run a follow-up window of the same length, then score.
| Criterion | Type | Evidence source |
|---|---|---|
| Oversharing review complete for the SharePoint sites in scope | Gate (pass/fail) | Permissions review report; findings closed or accepted by the data owner |
| Sensitivity labels and DLP policies in force for in-scope data | Gate (pass/fail) | Microsoft Purview policy report |
| Copilot interactions captured in the audit log | Gate (pass/fail) | Microsoft Purview Audit search for pilot users; retention matches your policy |
| Adoption | Scored | Microsoft Copilot usage report: active users rate over the last 28 days |
| Task time | Scored | Measured minutes on the task set, baseline against follow-up |
| Task quality | Scored | Defect rate from a sampled review against one rubric used in both windows |
| User sentiment | Scored, labeled self-reported | Pulse or survey results, reported separately |
Scoring rubric and decision rule
- Score each scored criterion 0, 1 or 2. 0 means below the threshold, 1 means it meets the threshold, 2 means it clearly exceeds it. The sponsor defines “clearly” before launch.
- Weights add up to 100 and are fixed before launch. The weighted score is the sum of each score multiplied by its weight, divided by 2.
- Any failed gate means adjust or stop, whatever the score.
- All gates pass and the weighted score meets the agreed bar: expand, within the scope written on the approval record.
- Otherwise adjust: change the task set, the training or the data scope, then run another follow-up window.
A user counts as active in Microsoft’s usage report only after an intentional action, such as submitting a prompt; opening the Copilot pane is not enough. Microsoft describes the Copilot Dashboard assisted-hours metric as an approximation, so the task-time score uses measured minutes from the pilot measurement sheet instead.
Synthetic task set
Five tasks with fixed inputs and a rubric each. They contain no client data; swap in your own tasks of the same shape.
- Finance: draft variance commentary from a provided budget-against-actual workbook. Rubric: figures match the workbook, drivers explained, no invented numbers.
- HR: answer an employee policy question from the handbook. Rubric: correct policy cited, correct eligibility, escalation when the handbook is silent.
- Sales: prepare a meeting brief from CRM notes and an email thread. Rubric: open items captured, actions assigned, no content from other customers’ files.
- Legal: summarize a contract against a clause checklist. Rubric: each clause found or marked missing, none invented, flagged for lawyer review.
- IT: turn an incident chat thread into a post-incident note. Rubric: timeline correct, root cause marked known or unknown, follow-ups listed.
The sales rubric doubles as an oversharing check: content from another customer’s files in the brief is a defect and a gate question.
The three gates cover the pilot’s own scope. Readiness of the whole tenant is scored before the pilot, under the eight-surface tenant AI readiness standard (TAR-8).
Approval record
The record names the pilot, the baseline and follow-up windows, each gate result with its evidence, the weighted score, the decision (expand, adjust or stop), the scope of any expansion (users, departments and data), conditions, and the sponsor, security owner and data owner who signed, with the date.
Download the Copilot pilot acceptance scorecard (CSV). The file holds the criteria, the rubric, the task set and a blank approval record. Its example values are synthetic and labeled as such; they are not client data. To work out the four measures behind the scores, see the Copilot ROI measurement framework. The gates are covered before launch in a Microsoft 365 Copilot readiness assessment.
Primary sources
- Microsoft Copilot usage report — Microsoft 365 admin — Microsoft Learn, accessed 2026-10-28
- Audit logs for Copilot and AI applications — Microsoft Learn, accessed 2026-10-28
- Connect to the Microsoft Copilot Dashboard for Microsoft 365 customers — Microsoft Learn, accessed 2026-10-28
Frequently Asked Questions
When should Copilot pilot exit criteria be set?
Before any pilot licenses are assigned. The sponsor writes down the thresholds, the weights and the decision rule, and signs them, so the result cannot be argued backward from whatever the pilot happened to show. The baseline window for task time and quality also has to run before licensing, or there is nothing to compare against.
Can a high adoption score override a failed safety check?
No. The oversharing review, the sensitivity label and DLP policies for in-scope data, and Copilot audit capture are pass/fail gates. If any gate fails, the decision is adjust or stop, whatever the weighted score says. A popular pilot that can surface overshared content is not ready to expand.
Should self-reported time savings count toward the score?
Report them, but on their own line. The time score comes from measured minutes on the defined task set, baseline window against follow-up window. Survey answers are labeled self-reported and never converted into hours. Microsoft describes the Copilot Dashboard assisted-hours metric as a broad approximation rather than a precise calculation, so treat it as directional.
Who should sign the pilot decision?
At minimum the business sponsor, the security owner and the data owner for the content Copilot can reach. The approval record names each one, the date, the decision (expand, adjust or stop), the scope of any expansion and the evidence attached: the measurement sheet, the usage report export and the gate reports.
Talk to a senior architect
A one-hour call with a senior EPC Group architect covers your pilot scope, the gates and how to set thresholds your finance team will accept. Email contact@epcgroup.net or call 888-381-9725.