Vendor Selection Scorecard
Score competing vendors on criteria and weights you control, then stress-test the result: if the winner flips when one weight moves, the decision rests on your weighting rather than your evidence, and the tool says so. Includes the cost lines missing from the proposal, the negotiation levers worth having, and an RFP scaffold.
You score the vendors. We supply the criteria, the weights, the evidence to demand instead of a claim, and a test of whether your answer actually holds. Nothing you enter leaves your browser.
Change the names, then rebuild the grid.
Score 0–10 per criterion on evidence you gathered, not on the demo. Adjust the weights to your situation — the defaults are a starting point, not a recommendation.
Why we do not score the vendors for you
The obvious version of this tool prints a table: Vendor A scores 82 on cost, 91 on security, 74 on integration. We will not publish that, for the same reason our benchmark carries no competitor rows. We have not run those products. Any score we assigned would be invented, and an invented comparative claim about a named third party is both worthless to you and actionable against us.
It would also be solving the wrong problem. The arithmetic was never the hard part of a vendor selection. The hard part is knowing which criteria decide the outcome, what each is worth, what evidence to demand rather than accept, and where each criterion is routinely gamed. That is what is below.
The stress test is the point
Most scorecards produce a winner that is an artefact of weights somebody picked in a meeting, then present it to two decimal places. This one moves each weight by a fifth in each direction and checks whether your winner survives.
If it does not, you are told which criterion flipped it. That is not a failure of the tool — it is the most useful output it can give you, because it means the decision currently rests on your weighting rather than on your evidence. Either gather better evidence on that criterion, or accept the options are genuinely close and decide on grounds you can defend. What you should not do is present a coin-toss as a clear recommendation, which is what a scorecard without this check quietly encourages.
Unscored is not neutral
A blank cell counts as zero and silently penalises that vendor. The tool tells you how many cells are filled and refuses to call a result defensible until the grid is complete.
Educational and informational. Not legal, financial or investment advice. Licensing terms are specific to your agreement — confirm anything commercial in writing with the vendor before relying on it.
The criteria, and what to demand instead of a claim
Default weights total 110 and are a starting point, not a recommendation — a regulated bank and a startup should not weight these the same way. Every vendor will answer yes to every question below. The column that matters is what they will show you.
Default weight 20
Total cost of ownership, three to five years
Year-one licence cost is the number in the proposal and the least useful number in the decision. The cost that matters accumulates in renewals, growth metrics, non-production, DR and the effort to leave.
Ask for. Give me year-one, year-three and year-five cost at our projected scale, with the uplift cap in writing. Then price the same thing at 150% of our current volume.
How it gets gamed. Discounts loaded into year one. The uplift at renewal is where it comes back, and it is rarely capped unless you ask.
Evidence. Per-employee metrics such as Oracle Java rise with hiring whether or not usage does, so the bill grows with a successful year /knowledge/licence-optimisation/
Default weight 12
Cost and feasibility of leaving
Exit cost is the single most under-weighted criterion in enterprise selection, and it is the one that determines your negotiating position at every future renewal. A product you cannot leave prices itself accordingly, and it does not need to be malicious about it.
Ask for. In what format is our data returned, over what period, at what cost, and who has done it? Ask for a reference who has actually migrated OFF the product.
How it gets gamed. "Full data export" that returns content without structure, permissions or history -- technically an export and practically a rebuild.
No base rate. No published figure on exit costs. Raised because it is structural: the party who cannot leave does not set the price.
Default weight 18
Security posture and evidence
Every vendor says yes to every security question in a sales cycle. The differentiator is not the answer but whether they will show you the artefact behind it.
Ask for. Current audit report under NDA, penetration test summary with remediation dates, and their breach notification window in the contract rather than in the brochure.
How it gets gamed. A certification scope that covers a different product, or a datacentre their software does not run in. Read the scope statement, not the badge.
Evidence. Vulnerability exploitation, 31%, is now the top initial access vector, so patch cadence in the supplier's own estate is a live question /research/cybersecurity-readiness/
Default weight 14
Permission model, if the product reads your content
For anything that indexes or searches enterprise content, this is the criterion most likely to cause a serious incident and the one least likely to appear on a scorecard. The failure is silent: a document served to somebody whose access was revoked, with no error anywhere.
Ask for. Are permissions evaluated when the user asks, or baked in when you index? If a user loses access at 09:00, when do they stop seeing the content? If an answer draws on five documents, what happens when the user may only see four?
How it gets gamed. "We respect source permissions" is true of both a safe design and a leaky one. The distinguishing question is the timing, not the intent.
Evidence. 30 of 30 adversarial permission cases pass on our own engine, including a group revoked mid-session -- the case an index-time filter serves until its next crawl /trust/benchmark/
Default weight 15
Integration with what you already run
Integration effort is where selection estimates are most wrong, and the error is always in the same direction. The demo connects to a clean system; yours has fifteen years of exceptions in it.
Ask for. Name three customers on our stack at our scale. What did integration actually take in elapsed weeks, and who did the work?
How it gets gamed. A connector list counts connectors, not depth. A connector that reads content but not permissions, or content but not history, is on the list.
Evidence. Reported blockers to AI reaching production are data access, ownership and integration rather than model capability /research/ai-agent-adoption/
Default weight 10
Data residency and regulatory fit
In regulated sectors this is frequently a pass or fail rather than a score, and it is discovered late because it is asked as a yes/no question that has a yes/no answer and a different reality behind it.
Ask for. Is residency contracted or technically enforced? What physically prevents a write to another region? Where do backups, logs and support access sit -- those are the three that escape the region.
How it gets gamed. Residency for primary data, with telemetry, backups and support tooling somewhere else entirely.
Evidence. The UAE has three non-overlapping regimes -- federal, DIFC and ADGM -- and which one applies is decided by trade licence, not office address /b2b/uae/
Default weight 8
Skills you have or can hire
A better product your team cannot operate loses to an adequate one they can. This criterion is usually scored on enthusiasm during selection and discovered properly during the first incident.
Ask for. How many people in our market hold this skill? What does the training path cost, and how long until someone is useful unsupervised?
How it gets gamed. Vendor training that certifies familiarity with the interface rather than competence in the platform.
No base rate. Market skill availability varies by geography and we have published no figures for it. Assess locally rather than accepting either party's impression.
Default weight 8
Vendor viability and ownership
Ownership changes change licensing models, and they do it faster than your migration can respond. This is not hypothetical in the current market.
Ask for. Who owns them, what has changed in their pricing model in the last three years, and what happens to our terms on a change of control?
How it gets gamed. Reassurance about roadmap from people with no authority over the acquisition.
Evidence. VMware moved to subscription-only with a general minimum order of 72 cores, raised from 16 in 2025 /knowledge/licence-optimisation/
Default weight 5
Support that is contractually real
Support quality is the criterion most often assessed from the sales experience, which is the one part of the relationship guaranteed not to resemble the rest of it.
Ask for. Put response and resolution targets in the contract with a remedy attached. Ask what proportion of P1s met the target last quarter.
How it gets gamed. Response time measured to an acknowledgement rather than to a human who can fix it.
No base rate. No published figures on vendor support performance.
The cost lines that are missing from the proposal
Year-one licence cost is the number in the proposal and the least useful number in the decision. Price all of these, at projected scale rather than today's.
| Line | Why it is missed |
|---|---|
| Licence or subscription, at projected scale | Model at growth, not at today's headcount or volume. |
| Non-production environments | Dev, test, UAT, SIT. The most common audit finding, and rarely in the proposal. |
| DR and standby capacity | Ask explicitly whether a passive site is licensed. Get it in writing naming your topology. |
| Integration and migration effort | Internal cost, not just the vendor's professional services quote. |
| Training and time to competence | Including the productivity dip nobody budgets for. |
| Infrastructure, storage and egress | Data egress is the line that surprises people at scale. |
| Renewal uplift across the term | Assume uncapped unless the cap is written down. |
| Cost to leave at end of term | Extraction, re-platforming, parallel running. Estimate it before you sign, not after. |
Negotiation levers, in the order they are worth having
Nearly all of these are obtainable during competition and close to impossible afterwards. That timing is the whole game: leverage is a function of when you ask, not how well.
| Lever | Why |
|---|---|
| A priced alternative | Leverage requires a credible walk-away, and credible means scoped, priced and time-boxed. Starting three months out tells the vendor you cannot move, and they know it before you do. |
| A written renewal uplift cap | The single highest-value clause in most agreements and among the easiest to obtain during competition. It is close to impossible to get afterwards. |
| The notice deadline, extracted and diarised | The date that matters is the notice deadline, not the renewal date, and it falls months earlier. Missing it renews the term at the vendor's number with no conversation. |
| The licensing metric itself | Negotiating unit price within a bad metric wins less than changing the metric. A per-employee model is a tax on hiring however good the discount is. |
| Audit terms and notice | Notice period, scope, frequency and who bears the cost. Negotiable during competition and never afterwards. |
| Data extraction obligations | Format, timeframe, assistance and cost, written into the contract. Without it, extraction is priced when you have the least leverage you will ever have. |
| Non-production entitlement in writing | Assumptions about free non-production use are the most common audit exposure. Get the position stated, not implied. |
| Change-of-control protection | Terms surviving an acquisition. The last two years have made this a live commercial question rather than boilerplate. |
RFP scaffold
| Section | The part people get wrong |
|---|---|
| Context and the problem | What is actually wrong today, in your words. Vendors write better proposals against a problem than against a feature list. |
| Scope, and explicit exclusions | What is out of scope matters more than what is in. Ambiguity here becomes a change request later. |
| Requirements, separated into must and should | If everything is a must, nothing is, and you will discover that during scoring. |
| Evidence required with each answer | State that unevidenced claims score zero. This single line changes the quality of what you receive. |
| Evaluation criteria and weights, published | Publishing the weights improves the responses and makes your own decision defensible afterwards. |
| Commercial model and the cost template | Give them your cost template so responses are comparable. Free-format pricing is not comparable and is sometimes not meant to be. |
| Security and data protection requirements | Ask for the artefacts, not the assurances. |
| Exit and transition obligations | In the RFP, so it is priced during competition rather than negotiated during a divorce. |
| References at your scale and on your stack | Including, if you can get it, one customer who left. |
One line changes the quality of what you receive more than any other: state in the RFP that unevidenced claims score zero.
It runs entirely in your browser and nothing you type is transmitted or stored. The output is indicative and derived from your own inputs, so it is only as good as they are. It is educational material, not professional, financial, legal or tax advice, and it is not an audit, a quote or a substitute for review by someone who can see your actual environment.