AI Vendor Evaluation Compliance Checklist

Introduction: The Vendor Decision That Defines Your AI Program

Your AI vendor selection may be the most consequential technology decision you make this year. Not because AI vendors are particularly risky — but because the compliance implications of using the wrong AI vendor with sensitive data can be severe and difficult to unwind.

In regulated industries, AI vendor selection is not primarily a feature comparison exercise. It is a compliance risk management exercise. The right vendor has the right technical capabilities and the right compliance posture for your regulatory environment. The wrong vendor — even with superior features — can create HIPAA violations, FedRAMP compliance gaps, fair lending risks, or SOC 2 failures that cost far more to remediate than the value the AI tool provides.

This checklist provides a structured framework for evaluating AI vendors for regulated industry use. It is organized into seven categories, each covering a critical dimension of vendor compliance assessment. Use it before making any AI vendor commitment that will involve regulated data.

TrustEdge, with 15+ years of compliance and risk management expertise through Jacobian Engineering, developed this checklist based on extensive AI vendor due diligence work across healthcare, financial services, legal, and government contracting.

How to Use This Checklist

Who should complete this checklist: IT Security, Compliance, Legal, and the business unit sponsor should each contribute to vendor evaluation. No single function has all the expertise required for a complete assessment.

When to use it: Complete this checklist before making a vendor selection decision for any AI tool that will process regulated data. For AI tools that will only process public or internal-only data, a lighter-weight version of this checklist may be sufficient.

How to score: For each item, mark: ✓ (vendor satisfies requirement), ✗ (vendor does not satisfy requirement), or ? (unclear — needs follow-up). Requirements marked as "Required" are non-negotiable for regulated data. Requirements marked "Recommended" represent best practices.

Category 1: Data Handling and Privacy

This category assesses how the vendor handles your data — the most fundamental compliance dimension for regulated industries.

1.1 Training Data Practices

[Required] Does the vendor explicitly prohibit using customer data to train or improve their AI models? Look for: A clear contractual prohibition, not just a vague commitment to "privacy." The prohibition should apply to all customer data, including queries, documents, and outputs. How to verify: Review the vendor's Terms of Service, Master Service Agreement, and Data Processing Agreement (or BAA for healthcare). Ask for written confirmation from the vendor if terms are ambiguous.

[Required] If the vendor uses customer data for any model improvement, is this opt-out (or opt-in)? Look for: At minimum, an opt-out mechanism that is easy to exercise and has been actually exercised. Opt-in is preferable for regulated data. How to verify: Ask vendor specifically: "Do you use our queries or data to improve your model? If yes, how do we opt out?"

[Recommended] Does the vendor have a clearly documented data lineage and provenance process for their training data? Look for: Documentation of where training data came from, how it was licensed, and whether it included any personal or regulated information. How to verify: Ask vendor for their AI model card or system card. Large vendors (OpenAI, Anthropic, Google) publish this information publicly; smaller vendors may not.

1.2 Data Residency and Storage

[Required] Does the vendor provide data residency options that meet your requirements? Look for: For US federal data, US-only processing. For EU data, EU data residency (or equivalent adequacy). For some highly regulated data, specific regions or jurisdictions may be required. How to verify: Review vendor's infrastructure documentation. Ask: "Where is data processed and stored? Can we restrict processing to [specific region]?"

[Required] Is data encrypted at rest? Look for: AES-256 or equivalent encryption for all customer data at rest, using customer-controlled or vendor-managed keys. Key management documentation. How to verify: Review vendor's security documentation (SOC 2 report, security FAQ). Ask for specific confirmation of encryption standards used.

[Required] Is data encrypted in transit? Look for: TLS 1.2 or higher for all data in transit. Certificate management practices. How to verify: Review vendor's security documentation. Can also be verified technically by examining TLS configuration.

[Recommended] Does the vendor support customer-managed encryption keys (CMEK)? Look for: The ability to use your own encryption keys, managed in your key management system, for data stored with the vendor. This provides an additional layer of protection — even if the vendor's infrastructure is compromised, your data cannot be decrypted without your keys. How to verify: Ask vendor: "Do you support customer-managed encryption keys? What key management systems do you support (AWS KMS, Azure Key Vault, etc.)?"

[Recommended] What is the vendor's data retention and deletion policy? Look for: Clear retention periods for customer data, with deletion upon customer request or contract termination. Verification that deletion is complete across all systems including backups. How to verify: Review vendor's data processing terms. Ask: "When we terminate our contract, how long before our data is completely deleted? Does deletion apply to backups?"

1.3 Data Access by Vendor Personnel

[Required] Does the vendor have documented policies limiting employee access to customer data? Look for: Need-to-know access controls for vendor employees. Prohibition on vendor employees accessing customer data except for specific, authorized support purposes. Background checks for employees with data access. How to verify: Review vendor's SOC 2 report (CC6 controls). Ask vendor for access control policy documentation.

[Recommended] Does the vendor log vendor employee access to customer data, and are those logs available to customers? Look for: Audit trails of vendor employee access to customer data, available to customers on request or through a self-service portal. How to verify: Ask vendor: "Do your employees access our data? If so, do you log that access and can we see those logs?"

Category 2: Security Certifications and Assessments

2.1 SOC 2

[Required for most regulated use cases] Does the vendor hold a current SOC 2 Type II report? Look for: SOC 2 Type II (not Type I — Type I is a point-in-time assessment, Type II covers a period of at least 6 months). The report should be recent (issued within the last 12 months). The report should cover Security (the CC criteria). Ideally also covers Availability and Confidentiality. How to verify: Request the vendor's SOC 2 Type II report. Review the report for the period covered, the trust service criteria covered, and the auditor's opinion. Review any exceptions noted.

[Recommended] Does the vendor's SOC 2 report specifically address AI system controls? Look for: Description of AI-specific controls in the system description. Evidence that AI model management, AI output quality, and AI vendor management are part of the SOC 2 scope. How to verify: Review the SOC 2 system description for AI-specific language.

2.2 FedRAMP (Government and Government Contractor Use Cases)

[Required for federal data] Does the vendor hold FedRAMP authorization at the appropriate impact level? Look for: FedRAMP Moderate authorization for most civilian agency use cases. FedRAMP High for systems handling highly sensitive federal data. How to verify: Check the FedRAMP Marketplace (marketplace.fedramp.gov) for the vendor's authorization status. Note: "In Process" is not authorized — the vendor must have an active Authority to Operate (ATO).

[Recommended] Is the vendor's FedRAMP authorization through the JAB (Joint Authorization Board) or agency authorization? Look for: JAB authorization (P-ATO) is the gold standard — it has been reviewed by the three major federal agencies and is reusable government-wide. Agency ATOs are valid but may not be recognized by all agencies. How to verify: The FedRAMP Marketplace shows the authorization type.

2.3 HIPAA Compliance (Healthcare Use Cases)

[Required for PHI] Will the vendor sign a HIPAA Business Associate Agreement? Look for: A vendor-provided BAA, or willingness to execute your organization's BAA. Note that a vendor's willingness to sign a BAA does not mean their BAA is adequate — see Category 4. How to verify: Ask vendor directly. Some vendors publish their BAA availability publicly.

[Recommended] Has the vendor undergone a HIPAA Security Risk Assessment conducted by a qualified third party? Look for: Documentation that the vendor has conducted a comprehensive HIPAA risk analysis. Third-party attestation is preferable to self-assessment. How to verify: Ask vendor for documentation of their HIPAA risk assessment.

2.4 Other Security Assessments

[Recommended] Does the vendor conduct annual third-party penetration testing? Look for: Annual penetration tests by an independent security firm. Scope should include AI-specific attack scenarios (prompt injection, model extraction) in addition to standard network and application testing. How to verify: Ask vendor for their pen testing cadence and scope. Request executive summary of most recent pen test (vendors typically will not share full reports).

[Recommended] Does the vendor have an ISO 27001 certification? Look for: Current ISO 27001 certificate covering the scope that includes your data. How to verify: Request certificate and verify with the certification body.

[Required] Does the vendor have a vulnerability management program? Look for: Defined SLAs for patching critical, high, medium, and low severity vulnerabilities. Process for communicating critical vulnerabilities to customers. How to verify: Ask vendor for their vulnerability management policy and patching SLAs.

Category 3: Availability and Business Continuity

[Required] What is the vendor's contractually committed SLA? Look for: Uptime SLA appropriate to the criticality of the AI system in your operations. 99.9% (8.7 hours downtime per year) is a reasonable minimum for business-critical applications. Verify that the SLA includes financial remedies for violations. How to verify: Review vendor's SLA documentation in the contract or service terms.

[Required] Does the vendor provide a publicly accessible status page? Look for: Real-time status page showing system availability and recent incidents. Historical incident data. How to verify: Ask vendor for their status page URL. Review it to assess incident history and transparency.

[Recommended] Does the vendor have a documented disaster recovery plan? Look for: RTO (Recovery Time Objective) and RPO (Recovery Point Objective) for disaster recovery. Evidence that the disaster recovery plan is tested regularly. How to verify: Ask vendor for their RTO and RPO. Ask whether DR testing results are available.

[Required] What happens to your data if the vendor goes out of business or is acquired? Look for: Contractual protections ensuring data access and portability if the vendor ceases operations. Data escrow arrangements for critical vendors. How to verify: Review vendor contract for data portability and termination provisions.

Category 4: Business Associate Agreement Quality (Healthcare)

This category applies specifically to AI vendors being considered for use with PHI.

[Required] Does the BAA explicitly prohibit using PHI for model training? Look for: An unambiguous prohibition on using PHI (or any customer data) to train, fine-tune, or improve the AI model. This must be in the BAA or attached agreement, not just in a policy document. How to verify: Read the BAA carefully. If the language is ambiguous, ask the vendor for clarification in writing.

[Required] Does the BAA require breach notification within HIPAA-required timeframes? Look for: Breach notification no later than 60 days after discovery. Earlier notification is better — 30 days is preferable. The BAA should require notification of the covered entity, who then notifies affected individuals. How to verify: Review the breach notification provision of the BAA. Note the specific timeframe.

[Required] Does the BAA address subcontractor/subprocessor obligations? Look for: A requirement that the vendor execute BAAs with all subcontractors who may receive or access PHI. Ideally a list of current subprocessors. How to verify: Review the BAA's subcontractor provision. Ask vendor for a current list of subprocessors.

[Required] Does the BAA provide for return or destruction of PHI upon termination? Look for: A commitment to return or destroy all PHI within a specified timeframe after contract termination. Certification of destruction. How to verify: Review the termination provision of the BAA.

[Recommended] Does the BAA provide audit rights? Look for: The right to audit the vendor's HIPAA compliance, or to receive audit reports from a qualified third party. How to verify: Review the audit rights provision of the BAA.

Category 5: AI-Specific Technical Capabilities

[Required] Does the vendor provide comprehensive audit logging of AI system use? Look for: Logs of all queries, responses, users, and timestamps. Logs accessible to the customer (not just to the vendor). Retention of logs for the required period. Logs exportable for integration with your SIEM or audit log management system. How to verify: Ask vendor for documentation of their audit logging capabilities. Request a sample log entry.

[Required] Does the vendor support role-based access control for AI system features? Look for: Ability to assign different permissions to different user roles — controlling who can query what data, who can administer the system, who can view audit logs. How to verify: Ask vendor for their access control model documentation.

[Recommended] Does the vendor provide explainability features for AI outputs? Look for: Citation of source documents in RAG outputs. Confidence scores. Explanation of reasoning where applicable. These are particularly important for regulated use cases where AI outputs must be auditable. How to verify: Request a demo of explainability features. Test with representative queries.

[Recommended] Does the vendor have documented procedures for handling prompt injection and other AI-specific security threats? Look for: Documented threat model for AI-specific attacks. Controls in place to detect and prevent prompt injection, jailbreaking, and model extraction. How to verify: Ask vendor for their AI security threat model. Ask what testing they do for adversarial inputs.

[Required] What is the vendor's process for communicating model updates? Look for: Advance notice before significant model changes. Changelogs for model updates. Option to remain on a specific model version for stability. This is important for regulated industries where model changes may require re-validation. How to verify: Ask vendor: "How do you communicate when your underlying AI model is updated? Can we freeze to a specific model version?"

Category 6: Contract and Legal

[Required] Are the vendor's data processing terms compatible with your regulatory requirements? Look for: Data processing terms (BAA, DPA, or equivalent) that: prohibit training on your data, specify appropriate data retention and deletion, include appropriate security requirements, provide breach notification meeting your regulatory requirements. How to verify: Have legal counsel review the data processing terms before signing. Use this checklist's BAA section for healthcare-specific review.

[Required] Does the vendor have adequate insurance? Look for: Cyber liability insurance at a coverage level appropriate to the vendor's size and the data they will hold. General liability and professional liability insurance. How to verify: Ask vendor for their certificate of insurance. Review coverage limits.

[Recommended] Does the vendor's contract include SLA remedies and limitation of liability terms acceptable to your organization? Look for: Financial remedies for SLA violations (service credits). Limitation of liability terms that are not so restrictive that they eliminate meaningful recourse for significant failures. How to verify: Have legal counsel review contract terms.

Category 7: Vendor Viability and Governance

[Required] Is the vendor financially stable? Look for: Established revenue, institutional funding, or other evidence of financial viability. Vendors that may not be around in 18 months are high-risk dependencies. How to verify: Review publicly available financial information (for public companies). For private companies, ask about funding, revenue, and runway. Check for news reports about financial difficulties.

[Recommended] Does the vendor have a dedicated security team? Look for: In-house security function with appropriate staffing and expertise. CISO or equivalent. Participation in security community (bug bounty programs, responsible disclosure policies). How to verify: Ask vendor about their security organization. Review their security leadership profiles on LinkedIn.

[Recommended] Does the vendor participate in industry compliance and security standards organizations? Look for: Membership in HITRUST, HL7 FHIR community, Cloud Security Alliance, or similar organizations. Participation signals engagement with the compliance and security community. How to verify: Ask vendor about their industry affiliations.

Scoring and Decision Framework

After completing the checklist, use this scoring approach:

Score of Required items:

Any Required item marked ✗: Do not use vendor for regulated data. Work with vendor to remediate before proceeding, or select an alternative vendor.
Any Required item marked ?: Escalate to legal/compliance for resolution before proceeding.

Score of Recommended items:

0-5 Recommended items satisfied: High risk. Consider alternatives.
6-10 Recommended items satisfied: Moderate risk. Address gaps with compensating controls.
11-15 Recommended items satisfied: Low risk. Proceed with standard ongoing monitoring.

Final decision process:

All Required items must be ✓ before proceeding
Document the assessment results and any gaps
Obtain approval from appropriate governance authority (see AI Governance Playbook for decision authority framework)
Document compensating controls for any gaps in Recommended items
Establish ongoing vendor monitoring cadence

After Vendor Selection: Ongoing Monitoring

Vendor assessment does not end at contract signing. Establish ongoing monitoring:

Annual: Re-complete this checklist for all high-risk AI vendors. Obtain and review updated SOC 2 report.

As triggered: Re-assess immediately if: vendor has a security incident, vendor changes their data handling terms, vendor is acquired, vendor loses a key security certification.

Continuous: Monitor vendor status page for incidents. Monitor news for vendor security developments. Review vendor security communications.

Conclusion

AI vendor due diligence is among the most important risk management activities for regulated organizations deploying AI. The investment in a structured vendor assessment process pays for itself many times over in avoided compliance violations, security incidents, and vendor relationship problems.

This checklist provides a starting point. Your organization may have additional requirements specific to your regulatory environment, client commitments, or organizational risk appetite. Work with your compliance and legal teams to customize this checklist for your specific context.

TrustEdge provides AI vendor due diligence services for regulated organizations — conducting thorough assessments of AI vendors' compliance posture, negotiating compliant BAAs and data processing terms, and providing ongoing vendor monitoring support.

Ready to build a rigorous AI vendor assessment program? Schedule a consultation with TrustEdge. Call (888) 555-EDGE or reach out through our website to speak with a compliance expert who can help you evaluate AI vendors for your specific regulatory environment.