FedRAMP and AI: Government Agency Guide

The Federal Government's AI Moment — and the Compliance Challenge It Creates

Federal agencies and the contractors that serve them are under enormous pressure to adopt artificial intelligence. Executive Order 14110 on "Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence," issued in October 2023, directed federal agencies to accelerate AI adoption while establishing guardrails for safety and security. OMB Memoranda M-24-10 and M-24-18 followed with specific requirements for federal AI use — including mandatory inventory of AI use cases, AI impact assessments, and designation of Chief AI Officers.

At the same time, the security requirements that govern federal IT systems — embodied in FedRAMP — remain among the most rigorous in the world. And those requirements do not bend for AI.

The result is a paradox: federal agencies and contractors are being pushed to adopt AI faster than ever, while the security authorization process required to do so compliantly is slower and more demanding than most commercial AI vendors have ever encountered.

TrustEdge, drawing on 15+ years of compliance engineering expertise from Jacobian Engineering, helps government agencies and contractors navigate this intersection — deploying AI that is both capable and genuinely compliant.

What Is FedRAMP? A Brief Primer

The Federal Risk and Authorization Management Program (FedRAMP) provides a standardized approach to security assessment, authorization, and continuous monitoring for cloud products and services used by federal agencies. It was created to replace the fragmented, agency-by-agency security review process that previously created enormous duplication of effort.

Under FedRAMP:

Cloud Service Providers (CSPs) seeking to sell to federal agencies must obtain a FedRAMP Authorization
Authorization requires implementation of security controls from NIST SP 800-53, with the specific controls depending on the impact level (Low, Moderate, or High)
Authorization is granted either by the FedRAMP Program Management Office (PMO) through the Joint Authorization Board (JAB), or by an individual agency (Agency Authorization)
Once authorized, the authorization package is made available to all agencies, enabling "authorize once, use many times"
Continuous Monitoring (ConMon) requirements ensure that FedRAMP-authorized systems maintain their security posture over time

The three FedRAMP baseline impact levels correspond to different data sensitivity:

FedRAMP Low: Systems where the loss of confidentiality, integrity, or availability would have limited adverse effects. ~125 security controls.
FedRAMP Moderate: Systems where the impact would be serious. ~325 security controls. Most civilian agency systems fall here.
FedRAMP High: Systems where the impact would be severe or catastrophic. ~421 security controls. Used for systems handling law enforcement data, emergency services, financial systems, and health systems.

The State of FedRAMP-Authorized AI

As of 2024, there are a relatively small number of AI services that hold FedRAMP authorization, and the landscape is evolving rapidly. Key authorized or in-authorization AI services include:

Microsoft Azure Government AI Services: Azure Cognitive Services and Azure OpenAI Service on Azure Government are available to agencies under Microsoft's FedRAMP High authorization for Azure Government. This is currently one of the most mature paths for agencies wanting to use foundation model AI capabilities in FedRAMP-compliant environments.

Google Cloud AI Platform (FedRAMP Moderate): Google's AI and machine learning services on Google Cloud are available under their FedRAMP Moderate authorization for certain configurations.

AWS GovCloud AI Services: Amazon Web Services GovCloud offers FedRAMP High authorization, and various AWS AI/ML services (SageMaker, Comprehend, Rekognition) are available within that boundary, though the specific services and configurations authorized vary.

Palantir AIP (Agency Authorization): Palantir's AI Platform has obtained agency-level FedRAMP authorization for several government customers.

Critically, many popular commercial AI tools — including the consumer versions of ChatGPT, Copilot, Claude, and others — are not FedRAMP authorized. Using these tools to process federal data, including Controlled Unclassified Information (CUI), may violate federal law and agency policy.

NIST SP 800-53 Controls Most Relevant to AI

The security control framework underlying FedRAMP — NIST SP 800-53 Rev 5 — was updated in 2020 to include new controls relevant to AI and automated decision-making. Key controls that agencies and contractors must understand for AI deployments include:

AC-3: Access Enforcement and AC-6: Least Privilege

AI systems must enforce approved authorizations for logical access to information and resources, consistent with applicable access control policies. This means:

Role-based access to AI capabilities based on job function and need-to-know
Separation of duties between AI system administrators, users, and auditors
Privileged access management for accounts with administrative access to AI systems

AU-2 and AU-12: Event Logging

All relevant events in AI systems must be logged and those logs must be reviewed. For AI systems, relevant events include:

User queries and AI responses (subject to storage and privacy constraints)
Model configuration changes
Training and fine-tuning operations
Integration with other systems
Error conditions and anomalies

CM-4: Impact Analysis and CM-6: Configuration Settings

Changes to AI systems — including updates to underlying models — must go through change management with impact analysis. Configuration settings must be documented and controlled.

This is particularly important when using third-party AI APIs. If OpenAI or Anthropic updates their model (even a minor version update), agencies and contractors using those APIs in FedRAMP-authorized systems must assess the impact of that change.

RA-5: Vulnerability Monitoring and SA-11: Developer Security Testing

AI systems and their dependencies must be monitored for vulnerabilities. This includes:

Regular scanning of AI application code and dependencies
Assessment of AI-specific vulnerabilities (prompt injection, model poisoning, adversarial inputs)
Penetration testing that includes AI-specific attack scenarios

NIST SP 800-218A (Secure Software Development Framework for Generative AI) provides specific guidance on security testing for AI systems that agencies should integrate into their authorization packages.

SI-12: Information Management and Retention

Information output from AI systems must be handled and retained in accordance with applicable laws and regulations. For federal systems, this includes Federal Records Act requirements, which may require retention of AI-generated documents as federal records.

SR-3: Supply Chain Controls

The AI supply chain — including foundation model providers, training data sources, and AI infrastructure — must be assessed as part of the system's supply chain risk management program. This is increasingly important as concerns about adversarial AI supply chain attacks have grown.

Controlled Unclassified Information (CUI) and AI

One of the most common compliance risks for federal contractors is the inadvertent processing of Controlled Unclassified Information (CUI) in non-authorized AI systems.

CUI is information the Government creates or possesses that requires safeguarding or dissemination controls pursuant to law, regulation, or Government-wide policy. CUI categories include:

Privacy Act data
Export controlled technical data
Law Enforcement Sensitive information
Proprietary business information submitted to the government
Pre-decisional deliberative information

NIST SP 800-171 (for non-federal systems processing CUI) and CMMC (Cybersecurity Maturity Model Certification) both require that CUI be protected on systems that meet specific security requirements. Using a non-CMMC-compliant AI tool to process CUI — for example, using commercial ChatGPT to draft a proposal based on CUI technical data — may violate DFARS 252.204-7012 and put a contractor at risk of losing contracts or facing False Claims Act liability.

The key rules for contractors:

Classify before you process: Know what type of data you are working with before using any AI tool
Check authorization status: Verify that any AI tool used for CUI is authorized for the appropriate data category
Use approved systems: CUI processing should occur in CMMC-compliant environments with AI tools that are part of the approved system boundary
Document exceptions: If using an AI tool for non-CUI work on a CUI contract, document the data handling procedures clearly

The FedRAMP Authorization Process for AI Products

If you are a technology company seeking to offer AI capabilities to federal agencies and need FedRAMP authorization, the process is substantial:

Step 1: System Boundary Definition

Define precisely what is included in the FedRAMP authorization boundary. For AI products, this includes:

AI model infrastructure (training and inference)
Application APIs
Data storage systems
Administrative interfaces
Integration points with other systems

Step 2: Categorization

Using FIPS 199 and NIST SP 800-60, determine the impact level for your system. Most enterprise AI products will seek FedRAMP Moderate; those serving law enforcement, health, or financial agencies may need FedRAMP High.

Step 3: Control Implementation

Implement all required controls from the applicable FedRAMP baseline. For AI systems, pay particular attention to:

Processing Integrity controls (ensuring AI outputs are accurate and reliable)
Supply Chain Risk Management controls (documenting model provenance)
Privacy controls (if handling PII or PHI)
Incident Response controls (including AI-specific incident scenarios)

Step 4: Documentation

Prepare the System Security Plan (SSP) and all supporting documentation. For AI systems, this includes:

Model documentation (what model is used, how it was trained, how it is updated)
Data flow diagrams showing how data moves through the AI system
Control implementation descriptions for all applicable controls
Policies and procedures for AI-specific processes

Step 5: Third-Party Assessment

A FedRAMP-authorized Third-Party Assessment Organization (3PAO) must assess your system. The assessment includes document review, interviews, and technical testing.

Step 6: Authorization and Continuous Monitoring

After receiving an Authority to Operate (ATO), maintain continuous monitoring — monthly vulnerability scans, annual penetration tests, and ongoing reporting of significant changes.

AI in the FedRAMP Authorization Package: What Assessors Are Looking For

Based on engagements with 3PAOs and agency authorizing officials, TrustEdge has identified the AI-specific areas where FedRAMP assessors are increasingly focused:

Model Explainability: For AI systems making decisions that affect individuals (benefits eligibility, risk scoring, etc.), assessors want to understand how the agency can explain and audit AI decisions.

Training Data Documentation: Where did the model come from? Who provided the training data? Was the data licensed appropriately? Was the data free of known biases?

Model Update Processes: How are model updates tested and authorized before deployment? How are changes communicated to the agency?

Privacy Impact Assessment: Has a Privacy Impact Assessment (PIA) been completed for AI systems processing PII? Does the PIA address AI-specific privacy risks?

Bias and Fairness Assessment: For AI systems affecting individuals, has the agency assessed whether the AI produces disparate outcomes for protected groups? NIST AI RMF (AI 600-1) guidance on bias is increasingly referenced in FedRAMP conversations.

Practical Recommendations for Agencies and Contractors

Based on extensive experience with FedRAMP compliance and AI deployment, TrustEdge recommends:

Establish an AI use case inventory — OMB M-24-10 requires this for agencies. Contractors should maintain their own inventory to ensure proper data classification and authorization assessment for each AI use.
Default to FedRAMP Moderate or High environments for sensitive AI workloads — If in doubt about whether data is sensitive enough to require a FedRAMP environment, err on the side of using one.
Require FedRAMP status as a vendor selection criterion — When evaluating AI vendors, FedRAMP authorization status should be a hard requirement for any product that will process federal data.
Plan for continuous monitoring of AI systems — FedRAMP ConMon requirements apply to AI systems within a FedRAMP boundary. Establish monitoring from day one.
Engage your Authorizing Official early — For novel AI use cases, engage with the agency's Authorizing Official before deployment. Getting an AO's informal buy-in early prevents costly rework later.

Conclusion: Compliant AI in Government Is Possible — With the Right Partner

Federal agencies and contractors do not have to choose between AI capability and compliance. The infrastructure for compliant government AI exists and is maturing rapidly. But navigating it requires deep expertise in both FedRAMP and the specific AI technologies being deployed.

TrustEdge brings that expertise to every engagement. Built on 15+ years of compliance and security work through Jacobian Engineering, our team has the FedRAMP knowledge, the AI technical depth, and the regulatory familiarity to help government organizations deploy AI that is both powerful and fully compliant.

Ready to move forward with compliant AI for your government organization? Schedule a consultation with TrustEdge. Call (888) 555-EDGE or reach out through our website to speak with a team that understands FedRAMP from the inside out.