Buying AI is harder than buying conventional software. The technology is new enough that most buyers have limited reference experience, vendors oversell with unusual enthusiasm, and the risks — hallucination, data leakage, model drift, unclear liability — do not fit neatly into the procurement templates built for databases and workflow tools. An enterprise that buys AI the way it buys office software will end up with impressive demos, vague contracts, and production systems it cannot fully explain or control.

This guide is for private-sector technology leaders — CIOs, CTOs, enterprise architects, and procurement partners — who need to select AI vendors and live with the consequences. It covers how to structure an AI request for proposal (RFP), how to score technical and commercial responses fairly, how to design demos and pilots that test what matters, and how to negotiate terms that protect you. (Public-sector organizations in Canada face additional procurement mechanics — fair-opportunity rules, trade-agreement thresholds, bilingual requirements — that are covered in our Canadian public-sector AI procurement guide.)

Start with the problem, not the product

An RFP (request for proposal) is a formal invitation to vendors to propose a solution to a defined problem, with responses evaluated against stated criteria. The most common failure in AI procurement happens before the RFP is written: the buying team frames the requirement as a product category (“we need a generative AI platform”) rather than a business outcome (“we need to cut commercial contract review time by half while maintaining our legal team’s approval standard”).

Outcome-based framing matters more for AI than for conventional software because vendor offerings differ wildly in architecture, capability, and risk. Two vendors can both sell “an AI document-processing solution” built on completely different architectures. If your RFP describes the product you think you want, you will get proposals for different things and no fair way to compare them. If it describes the outcome you need — inputs, throughput, accuracy targets, integration points, constraints — vendors must explain how they achieve it, and their answers become comparable.

Before writing anything, define three things: the decision the system will support or automate, the data it will touch, and the harm if it gets things wrong. A marketing copy generator, a customer-service assistant, and an underwriting co-pilot are all “AI,” but their risk profiles, data needs, and evaluation criteria are completely different. Our AI governance guide covers how to classify AI risk systematically; the risk tier you land on should directly shape your RFP’s weighting and terms.

Writing requirements that separate contenders from pretenders

AI RFP requirements typically fall into four groups, and each group plays a different role in evaluation:

Functional requirements describe what the system must do: ingest these document formats, integrate with these systems, support these languages, process this volume. These are the capabilities that map to your outcome.
Technical requirements describe how the system must behave architecturally: deployment options (cloud, on-premise, private instance), API standards, authentication, latency and throughput targets, data retention behavior.
Governance and risk requirements describe the controls you need: audit logging, human-in-the-loop options, explainability support, data usage restrictions, model versioning, incident response.
Commercial requirements describe the business terms: pricing model, term length, service levels, support model, exit assistance.

The critical discipline is separating must-have (pass/fail) criteria from scored criteria. Must-haves are the non-negotiables — things that disqualify a vendor outright if absent. Data residency in a specific jurisdiction, no use of your data for model training, and support for your identity provider are typical AI must-haves. Scored criteria are the differentiators — accuracy on your test set, depth of integration, quality of support — where vendors legitimately vary and you want to reward the best.

Most AI RFPs fail by putting too much in the scored column and too little in the pass/fail column, or vice versa. A practical rule: a requirement is a must-have only if failing it would stop the project regardless of how good everything else is. Everything else gets scored.

Structuring the scoring model

The scoring model is where your priorities become numbers. A common and defensible structure for AI purchases weights four dimensions:

Dimension	Typical weight	What it evaluates
Technical solution	30–40%	Functional fit, architecture, performance, scalability
Risk and governance	20–30%	Security, data handling, model controls, compliance
Commercial	15–25%	Total cost, pricing clarity, value for money
Vendor capability	10–15%	Financial health, references, delivery capability, roadmap

The weights are the strategy. An organization buying a customer-facing assistant should weight risk and governance heavily — the reputational exposure of a bad answer is the real cost. An organization buying an internal developer-productivity tool might weight technical solution and commercial terms more. Set the weights before proposals arrive, document the rationale, and do not adjust them after you have seen pricing — re-weighting after the fact is how incumbent favoritism and unconscious bias smuggle themselves into defensible-looking math.

Within each dimension, define scored criteria with explicit evidence requirements. “Describe your approach to model accuracy” invites marketing prose. “Provide accuracy metrics on a benchmark of your choice, describe the evaluation methodology, and provide three customer references where accuracy was measured in production” invites comparable facts. Every scored question should tell the vendor what evidence you expect and how much weight the answer carries.

For a reusable framework you can apply to any technology purchase — not just AI — see our vendor selection scorecard guide, which covers criteria design, calibration, and audit-ready documentation in depth.

Assembling the evaluation committee

AI purchases touch more stakeholders than conventional software, and the committee should reflect that. At minimum, include: the business owner who owns the outcome, an enterprise or solution architect who can judge technical claims, a security and privacy reviewer, a data governance or legal reviewer for data-rights terms, and a procurement lead who owns process integrity. For high-risk use cases — anything affecting customers, employees’ livelihoods, or regulated decisions — add the relevant compliance function early, not at contract review.

Two rules keep committees honest. First, every evaluator scores independently before any group discussion; group scoring sessions should start from written individual scores, not from whoever speaks first. Second, the committee needs at least one member who can read a vendor’s technical claims skeptically — someone who knows what questions to ask when a vendor says “our model is fine-tuned on your industry data” or “we use retrieval to eliminate hallucinations.” If that expertise does not exist in-house, engage it externally before the RFP goes out, not after the contract is signed.

Calibrating scores

A scoring scale without calibration is decoration. Define what each score means in terms of evidence, and do it in writing before evaluation begins. A five-point scale might read: 0 — no response or non-compliant; 1 — response provided but does not meet the requirement; 2 — partially meets, with material gaps; 3 — meets the requirement with supporting evidence; 4 — exceeds the requirement with strong evidence or proven differentiators.

Run a calibration exercise early: have every evaluator score the same two or three vendor responses (or the same sample answers) for one criterion, then compare. If one evaluator gives a 2 and another gives a 4 on the same evidence, the scale — not the vendor — is the problem, and it is far cheaper to fix the scale in week one than to explain inconsistent scoring to an auditor in month six. For AI evaluations specifically, watch for the demo halo effect: a polished live demo inflates scores on unrelated criteria like security posture or support quality. Calibrate by requiring evidence for every score — if an evaluator cannot point to the page or answer that justifies the number, the number does not stand.

Demos and proofs of concept

Written proposals establish claims; demos and pilots test them. Design both deliberately, because an undirected vendor demo will show you the product’s best thirty minutes, not the product you will operate.

Demo scripts should be scenario-based and identical for every vendor. Give each vendor the same business scenario, the same sample data, and the same set of tasks to complete live. Score against the script, not against presentation quality. Include at least one scenario the vendor did not prepare for — a malformed input, an edge-case question, a request the system should refuse — because how a system handles failure is more informative than how it handles the happy path. For AI specifically, test the behaviors vendors least want you to see: feed the system incorrect context and watch whether it hallucinates confidently or admits uncertainty; ask about topics outside its scope and check the refusal behavior; probe whether user data visibly leaks between sessions.

A proof of concept (PoC) — a limited trial of the technology in your environment, on your data, against your success criteria — is the single most valuable evaluation step for AI purchases, and the one most often skipped for speed. Structure it as a miniature production engagement: define success criteria in advance (accuracy thresholds, latency targets, user acceptance), run it on representative data (not the vendor’s demo data), involve the actual end users, and time-box it to four to eight weeks. The PoC agreement should state explicitly who owns the data and outputs generated during the trial, that the vendor may not use your data for training, and what happens to the data when the PoC ends. Vendors that resist these terms during a trial will resist them during a contract — treat resistance as information.

Red flags in vendor responses

Experience across AI evaluations surfaces a consistent set of warning signs:

Vaporware and roadmap selling. The response describes capabilities “releasing next quarter” as if they exist today. Require every claimed capability to be demonstrated in the current product; score roadmap items separately and discount them heavily.
Vague training-data claims. “Trained on high-quality data” or “fine-tuned on millions of documents” without specifics on data sources, licensing, or consent. Press for the provenance: what data, licensed how, with what rights. Unclear data provenance is a copyright and reputational risk that transfers to you at deployment.
Hallucination hand-waving. “Our retrieval layer eliminates hallucinations” or “our model doesn’t make things up.” No current system eliminates confabulation; serious vendors describe their mitigation and measurement approach — evaluation harnesses, confidence thresholds, human review workflows — and show their error rates. Absolute claims about AI accuracy are a credibility filter.
Lock-in architecture. Proprietary data formats, no API for bulk export, models that only run on the vendor’s infrastructure, pricing that punishes partial exit. Map the exit path before you sign; if the vendor cannot describe how you would leave, assume you cannot.
Security theater. SOC 2 mentioned without a report provided, “military-grade encryption” without specifics, penetration tests referenced without dates or scope. Real security posture comes with artifacts you can review — see our vendor security review guide for how to assess them.
Liability deflection. Terms that disclaim all responsibility for the system’s outputs while marketing those outputs as reliable. If the vendor will not stand behind the product’s behavior in any form, neither should you.

Commercial terms that protect you

AI contracts need clauses that conventional software agreements often lack or understate. Negotiate these before selection is final, while you still have competitive leverage:

Data rights. Your data is used only to provide the service to you — never for training the vendor’s models, never shared with other customers, never retained after termination beyond a defined wind-down period. Get this in the contract, not in a sales email.
Model portability and exit. If the solution includes custom fine-tuning or configuration built on your data, define who owns those artifacts and in what form you receive them at exit.
Termination rights. Include termination for convenience with a reasonable notice period, and termination for cause triggered by defined failures — repeated accuracy degradation, security incidents, or regulatory non-compliance. Pair these with exit assistance obligations: data export in usable formats, transition support, defined timelines.
Liability for AI failures. Standard limitation-of-liability clauses were written for software bugs, not for systems that generate plausible-sounding wrong answers to your customers. Push for carve-outs: the vendor should retain meaningful liability for data breaches, IP infringement arising from model outputs, and failures of the specific accuracy or safety commitments it made in the proposal.
Model and data change controls. Foundation models update frequently. Require notice of material model changes, the right to test before a new version goes live in your environment, and rollback rights. An unannounced model update that changes your system’s behavior is an operational risk you should be able to veto.
Audit and transparency. Rights to audit security controls, to receive incident notifications within defined timeframes, and to obtain documentation on training data, evaluation results, and known limitations. If the vendor’s system makes consequential decisions, your regulators — and your board — will expect you to have these answers.

Pilot-to-contract conversion

A successful PoC does not automatically become a successful production deployment. Plan the conversion explicitly: define the production success criteria separately from the PoC criteria (a PoC proves the technology works; production proves it works at scale, integrated with your systems, operated by your people), negotiate the production contract terms during the PoC — not after, when your leverage is gone — and set a decision gate with a real alternative. The alternative to converting the pilot is walking away; if walking away is not a credible option because the business has already committed to the outcome, the vendor knows it and will price accordingly.

Budget for integration, training, monitoring infrastructure, and human oversight — costs the vendor’s proposal will not include, since the vendor does not bear them.

Putting it into practice

Good AI procurement is a discipline, not a document: define the outcome, separate non-negotiables from differentiators, weight what matters, calibrate your scorers, test with your data, and negotiate protections while you still have leverage. Organizations that invest this rigor up front spend far less time renegotiating — or unwinding — bad AI contracts later.

If your organization is preparing a significant AI purchase — drafting requirements, structuring an evaluation, or reviewing terms a vendor has proposed — an experienced outside perspective can sharpen the process considerably. Our technology advisory engagements include procurement support and vendor evaluation assistance, and our capabilities cover enterprise AI architecture, AI governance, and technology strategy. To discuss your upcoming AI purchase, contact us.

Evaluating AI Vendor Proposals: RFP Design and Scoring for Enterprise Buyers