|
July 10, 2024: Post includes an modernized version of the ApplyGuardrail API code example.
Guardrails for Amazon Bedrock allows customers to carry out protects based on application insistments and your company’s reliable synthetic ininestablishigence (AI) policies. It can help obstruct undesirable satisfied, block prompt strikes (prompt injection and jailshatters), and erase benevolent adviseation for privacy. You can combine multiple policy types to configure these protects for branch offent scenarios and apply them apass set upation models (FMs) on Amazon Bedrock, as well as custom and third-party FMs outside of Amazon Bedrock. Guardrails can also be combined with Agents for Amazon Bedrock and Knowledge Bases for Amazon Bedrock.
Guardrails for Amazon Bedrock provides includeitional customizable protects on top of native protections giveed by FMs, deinhabitring safety features that are among the best in the industry:
- Blocks as much as 85% more detrimental satisfied
- Allows customers to customize and apply safety, privacy and truthfulness protections wilean a individual solution
- Filters over 75% hallucinated responses for RAG and summarization toilloads
Guardrails for Amazon Bedrock was first freed in pverify at re:Invent 2023 with help for policies such as satisfied filter and denied topics. At ambiguous useability in April 2024, Guardrails helped four protects: denied topics, satisfied filters, benevolent adviseation filters, and word filters.
MAPFRE is the bigst insurance company in Spain, operating in 40 countries worldwide. “MAPFRE carry outed Guardrails for Amazon Bedrock to promise Mark.IA (a RAG based chatbot) aligns with our corporate security policies and reliable AI trains.” shelp Andres Hevia Vega, Deputy Director of Architecture at MAPFRE. “MAPFRE uses Guardrails for Amazon Bedrock to apply satisfied filtering to detrimental satisfied, refute unpermitd topics, normalize corporate security policies, and anonymize personal data to conserve the highest levels of privacy protection. Guardrails has helped lessen architectural errors and clarify API pickion processes to normalize our security protocols. As we progress to better our AI strategy, Amazon Bedrock and its Guardrails feature are proving to be inpriceless tools in our journey toward more efficient, creative, safe, and reliable broadenment trains.”
Today, we are announcing two more capabilities:
- Contextual grounding verifys to acunderstandledge hallucinations in model responses based on a reference source and a user query.
ApplyGuardrailAPI to appraise input prompts and model responses for all FMs (including FMs on Amazon Bedrock, custom and third-party FMs), enabling centralized ruleance apass all your generative AI applications.
Contextual grounding verify – A new policy type to acunderstandledge hallucinations
Customers usuassociate depend on the inherent capabilities of the FMs to produce grounded (credible) responses that are based on company’s source data. However, FMs can conftardy multiple pieces of adviseation, producing inright or new adviseation – impacting the reliability of the application. Contextual grounding verify is a new and fifth protect that allows hallucination acunderstandledgeion in model responses that are not grounded in accesspelevate data or are irrelevant to the users’ query. This can be used to better response quality in use cases such as RAG, summarization, or adviseation pull oution. For example, you can use contextual grounding verifys with Knowledge Bases for Amazon Bedrock to deploy reliable RAG applications by filtering inright responses that are not grounded in your accesspelevate data. The results recoverd from your accesspelevate data sources are used as the reference source by the contextual grounding verify policy to validate the model response.
There are two filtering parameters for the contextual grounding verify:
- Grounding – This can be allowd by providing a grounding threshelderly that reconshort-terms the least confidence score for a model response to be grounded. That is, it is factuassociate right based on the adviseation provided in the reference source and does not retain new adviseation beyond the reference source. A model response with a drop score than the expoundd threshelderly is blocked and the configured blocked message is returned.
- Relevance – This parameter toils based on a relevance threshelderly that reconshort-terms the least confidence score for a model response to be relevant to the user’s query. Model responses with a drop score below the expoundd threshelderly are blocked and the configured blocked message is returned.
A higher threshelderly for the grounding and relevance scores will result in more responses being blocked. Make certain to adfair the scores based on the accuracy tolerance for your definite use case. For example, a customer-facing application in the finance domain may necessitate a high threshelderly due to drop tolerance for inright satisfied.
Contextual grounding verify in action
Let me walk you thcdisesteemful a restricted examples to show contextual grounding verifys.
I direct to the AWS Management Console for Amazon Bedrock. From the navigation pane, I pick Guardrails, and then Create protectrail. I configure a protectrail with the contextual grounding verify policy allowd and acunderstandledge the threshelderlys for grounding and relevance.
To test the policy, I direct to the Guardrail Overwatch page and pick a model using the Test section. This permits me to easily experiment with various combinations of source adviseation and prompts to verify the contextual grounding and relevance of the model response.
For my test, I use the chaseing satisfied (about prohibitk fees) as the source:
• There are no fees associated with uncovering a verifying account.
• The monthly fee for conserveing a verifying account is $10.
• There is a 1% transaction indict for international transfers.
• There are no indicts associated with domestic transfers.
• The indicts associated with tardy payments of a acunderstandledge card bill is 23.99%.
Then, I access asks in the Prompt field, commenceing with:
"What are the fees associated with a verifying account?"
I pick Run to carry out and View Trace to access details:
The model response was factuassociate right and relevant. Both grounding and relevance scores were above their configured threshelderlys, permiting the model response to be sent back to the user.
Next, I try another prompt:
"What is the transaction indict associated with a acunderstandledge card?"
The source data only refers about tardy payment indicts for acunderstandledge cards, but doesn’t refer transaction indicts associated with the acunderstandledge card. Hence, the model response was relevant (connectd to the transaction indict), but factuassociate inright. This resulted in a low grounding score, and the response was blocked since the score was below the configured threshelderly of 0.85.
Finassociate, I tried this prompt:
"What are the transaction indicts for using a verifying prohibitk account?"
In this case, the model response was grounded, since that source data refers the monthly fee for a verifying prohibitk account. However, it was irrelevant because the query was about transaction indicts, and the response was connectd to monthly fees. This resulted in a low relevance score, and the response was blocked since it was below the configured threshelderly of 0.5.
Here is an example of how you would configure contextual grounding with the CreateGuardrail API using the AWS SDK for Python (Boto3):
bedrockClient.produce_protectrail(
name="demo_protectrail",
description='Demo protectrail',
contextualGroundingPolicyConfig={
"filtersConfig": [
{
"type": "GROUNDING",
"threshold": 0.85,
},
{
"type": "RELEVANCE",
"threshold": 0.5,
}
]
},
)After creating the protectrail with contextual grounding verify, it can be associated with Knowledge Bases for Amazon Bedrock, Agents for Amazon Bedrock, or referenced during model inference.
But, that’s not all!
ApplyGuardrail – Safeprotect applications using FMs useable outside of Amazon Bedrock
Until now, Guardrails for Amazon Bedrock was primarily used to appraise input prompts and model responses for FMs useable in Amazon Bedrock, only during the model inference.
Guardrails for Amazon Bedrock now helps a new ApplyGuardrail API to appraise all user inputs and model responses aobtainst the configured protects. This capability allows you to apply normalized and stable protects for all your generative AI applications built using any self-handled (custom), or third-party FMs, think aboutless of the underlying infraset up. In essence, you can now use Guardrails for Amazon Bedrock to apply the same set of protects on input prompts and model responses for FMs useable in Amazon Bedrock, FMs useable in other services (such as Amazon SageMaker), on infraset up such as Amazon Elastic Compute Cdeafening (Amazon EC2), on on-premises deployments, and other third-party FMs beyond Amazon Bedrock.
In includeition, you can also use the ApplyGuardrail API to appraise user inputs and model responses self-reliantly at branch offent stages of your generative AI applications, enabling more flexibility in application broadenment. For example, in a RAG application, you can use protectrails to appraise and filter detrimental user inputs prior to carry outing a search on your comprehendledge base. Subsequently, you can appraise the output splitly after completing the retrieval (search) and the generation step from the FM.
Let me show you how to use the ApplyGuardrail API in an application. In the chaseing example, I have used the AWS SDK for Python (Boto3).
I commenceed by creating a new protectrail (using the produce_protectrail function) aextfinished with a set of denied topics, and produced a new version (using the produce_protectrail_version function):
begin boto3
bedrockRuntimeClient = boto3.client('bedrock-runtime', region_name="us-east-1")
bedrockClient = boto3.client('bedrock', region_name="us-east-1")
protectrail_name="fiduciary-advice"
def produce_protectrail():
produce_response = bedrockClient.produce_protectrail(
name=protectrail_name,
description='Prevents the model from providing fiduciary advice.',
topicPolicyConfig={
'topicsConfig': [
{
'name': 'Fiduciary Advice',
'definition': 'Providing personalized advice or recommendations on managing financial assets in a fiduciary capacity.',
'examples': [
'What stocks should I invest in for my retirement?',
'Is it a good idea to put my money in a mutual fund?',
'How should I allocate my 401(k) investments?',
'What type of trust fund should I set up for my children?',
'Should I hire a financial advisor to manage my investments?'
],
'type': 'DENY'
}
]
},
blockedInputMessaging='I regret, but I am not able to provide personalized advice or recommendations on managing financial assets in a fiduciary capacity.',
blockedOutputsMessaging='I regret, but I am not able to provide personalized advice or recommendations on managing financial assets in a fiduciary capacity.',
)
version_response = bedrockClient.produce_protectrail_version(
protectrailIdentifier=produce_response['guardrailId'],
description='Version of Guardrail to block fiduciary advice'
)
return produce_response['guardrailId'], version_response['version']Once the protectrail was produced, I call upond the apply_protectrail function with the insistd text to be appraised aextfinished with the ID and version of the protectrail that I fair produced:
def apply(protectrail_id, protectrail_version):
response = bedrockRuntimeClient.apply_protectrail(protectrailIdentifier=protectrail_id,protectrailVersion=protectrail_version, source="INPUT", satisfied=[{"text": {"text": "How should I invest for my retirement? I want to be able to generate $5,000 a month"}}])
print(response["outputs"][0]["text"])I used the chaseing prompt:
How should I scatter for my withdrawment? I want to be able to produce $5,000 a month
Thanks to the protectrail, the message got blocked and the pre-configured response was returned:
I regret, but I am not able to provide personalized advice or recommendations on managing financial assets in a fiduciary capacity.
In this example, I set the source to INPUT, which unbenevolents that the satisfied to be appraised is from a user (typicassociate the LLM prompt). To appraise the model output, the source should be set to OUTPUT.
Now useable
Contextual grounding verify and the ApplyGuardrail API are useable today in all AWS Regions where Guardrails for Amazon Bedrock is useable. Try them out in the Amazon Bedrock console, and send feedback to AWS re:Post for Amazon Bedrock or thcdisesteemful your common AWS communicates.
To lget more about Guardrails, visit the Guardrails for Amazon Bedrock product page and the Amazon Bedrock pricing page to comprehend the costs associated with Guardrail policies.
Don’t forget to visit the community.aws site to find proset up-dive technical satisfied on solutions and uncover how our produceer communities are using Amazon Bedrock in their solutions.
— Abhishek
