We recommend new projects start with resources from the AWS provider.
published on Wednesday, Nov 18, 2026 by Pulumi
We recommend new projects start with resources from the AWS provider.
published on Wednesday, Nov 18, 2026 by Pulumi
Resource Type definition for AWS::BedrockAgentCore::Evaluator - Creates a custom evaluator for agent quality assessment using LLM-as-a-Judge, code-based, or derived evaluator configurations. Derived evaluators run an existing built-in or third-party evaluator’s logic on a model that you supply.
Using getEvaluator
Two invocation forms are available. The direct form accepts plain arguments and either blocks until the result value is available, or returns a Promise-wrapped result. The output form accepts Input-wrapped arguments and returns an Output-wrapped result.
function getEvaluator(args: GetEvaluatorArgs, opts?: InvokeOptions): Promise<GetEvaluatorResult>
function getEvaluatorOutput(args: GetEvaluatorOutputArgs, opts?: InvokeOutputOptions): Output<GetEvaluatorResult>def get_evaluator(evaluator_arn: Optional[str] = None,
opts: Optional[InvokeOptions] = None) -> GetEvaluatorResult
def get_evaluator_output(evaluator_arn: pulumi.Input[Optional[str]] = None,
opts: Optional[InvokeOutputOptions] = None) -> Output[GetEvaluatorResult]func LookupEvaluator(ctx *Context, args *LookupEvaluatorArgs, opts ...InvokeOption) (*LookupEvaluatorResult, error)
func LookupEvaluatorOutput(ctx *Context, args *LookupEvaluatorOutputArgs, opts ...InvokeOption) LookupEvaluatorResultOutput> Note: This function is named LookupEvaluator in the Go SDK.
public static class GetEvaluator
{
public static Task<GetEvaluatorResult> InvokeAsync(GetEvaluatorArgs args, InvokeOptions? opts = null)
public static Output<GetEvaluatorResult> Invoke(GetEvaluatorInvokeArgs args, InvokeOptions? opts = null)
public static Output<GetEvaluatorResult> Invoke(GetEvaluatorInvokeArgs args, InvokeOutputOptions opts)
}public static CompletableFuture<GetEvaluatorResult> getEvaluator(GetEvaluatorArgs args, InvokeOptions options)
public static Output<GetEvaluatorResult> getEvaluator(GetEvaluatorArgs args, InvokeOptions options)
public static Output<GetEvaluatorResult> getEvaluator(GetEvaluatorArgs args, InvokeOutputOptions options)
fn::invoke:
function: aws-native:bedrockagentcore:getEvaluator
arguments:
# arguments dictionarydata "aws-native_bedrockagentcore_get_evaluator" "name" {
# arguments
}The following arguments are supported:
- Evaluator
Arn string - The Amazon Resource Name (ARN) of the evaluator.
- Evaluator
Arn string - The Amazon Resource Name (ARN) of the evaluator.
- evaluator_
arn string - The Amazon Resource Name (ARN) of the evaluator.
- evaluator
Arn String - The Amazon Resource Name (ARN) of the evaluator.
- evaluator
Arn string - The Amazon Resource Name (ARN) of the evaluator.
- evaluator_
arn str - The Amazon Resource Name (ARN) of the evaluator.
- evaluator
Arn String - The Amazon Resource Name (ARN) of the evaluator.
getEvaluator Result
The following output properties are available:
- Created
At string - The timestamp when the evaluator was created.
- Description string
- The description of the evaluator.
- Evaluator
Arn string - The Amazon Resource Name (ARN) of the evaluator.
- Evaluator
Config Pulumi.Aws Native. Bedrock Agent Core. Outputs. Evaluator Config - The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
- Evaluator
Id string - The unique identifier of the evaluator.
- Evaluator
Type Pulumi.Aws Native. Bedrock Agent Core. Evaluator Type - The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
- Kms
Key stringArn - The ARN of the KMS key used to encrypt evaluator data.
- Level
Pulumi.
Aws Native. Bedrock Agent Core. Evaluator Level - The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
- Provider
Pulumi.
Aws Native. Bedrock Agent Core. Evaluator Provider - Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
- Status
Pulumi.
Aws Native. Bedrock Agent Core. Evaluator Status - The current status of the evaluator.
-
List<Pulumi.
Aws Native. Outputs. Tag> - A list of tags to assign to the evaluator.
- Updated
At string - The timestamp when the evaluator was last updated.
- Created
At string - The timestamp when the evaluator was created.
- Description string
- The description of the evaluator.
- Evaluator
Arn string - The Amazon Resource Name (ARN) of the evaluator.
- Evaluator
Config EvaluatorConfig - The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
- Evaluator
Id string - The unique identifier of the evaluator.
- Evaluator
Type EvaluatorType - The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
- Kms
Key stringArn - The ARN of the KMS key used to encrypt evaluator data.
- Level
Evaluator
Level - The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
- Provider
Evaluator
Provider - Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
- Status
Evaluator
Status - The current status of the evaluator.
- Tag
- A list of tags to assign to the evaluator.
- Updated
At string - The timestamp when the evaluator was last updated.
- created_
at string - The timestamp when the evaluator was created.
- description string
- The description of the evaluator.
- evaluator_
arn string - The Amazon Resource Name (ARN) of the evaluator.
- evaluator_
config object - The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
- evaluator_
id string - The unique identifier of the evaluator.
- evaluator_
type "Builtin" | "ThirdParty" | "Custom" | "Custom Code" | "Custom Derived" - The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
- kms_
key_ stringarn - The ARN of the KMS key used to encrypt evaluator data.
- level "TOOL_CALL" | "TRACE" | "SESSION"
- The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
- provider
"AWS" | "Deep
Eval" | "Auto Eval" | "Custom" - Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
- status "ACTIVE" | "CREATING" | "CREATE_FAILED" | "UPDATING" | "UPDATE_FAILED" | "DELETING"
- The current status of the evaluator.
- list(object)
- A list of tags to assign to the evaluator.
- updated_
at string - The timestamp when the evaluator was last updated.
- created
At String - The timestamp when the evaluator was created.
- description String
- The description of the evaluator.
- evaluator
Arn String - The Amazon Resource Name (ARN) of the evaluator.
- evaluator
Config EvaluatorConfig - The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
- evaluator
Id String - The unique identifier of the evaluator.
- evaluator
Type EvaluatorType - The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
- kms
Key StringArn - The ARN of the KMS key used to encrypt evaluator data.
- level
Evaluator
Level - The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
- provider
Evaluator
Provider - Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
- status
Evaluator
Status - The current status of the evaluator.
- List<Tag>
- A list of tags to assign to the evaluator.
- updated
At String - The timestamp when the evaluator was last updated.
- created
At string - The timestamp when the evaluator was created.
- description string
- The description of the evaluator.
- evaluator
Arn string - The Amazon Resource Name (ARN) of the evaluator.
- evaluator
Config EvaluatorConfig - The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
- evaluator
Id string - The unique identifier of the evaluator.
- evaluator
Type EvaluatorType - The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
- kms
Key stringArn - The ARN of the KMS key used to encrypt evaluator data.
- level
Evaluator
Level - The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
- provider
Evaluator
Provider - Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
- status
Evaluator
Status - The current status of the evaluator.
- Tag[]
- A list of tags to assign to the evaluator.
- updated
At string - The timestamp when the evaluator was last updated.
- created_
at str - The timestamp when the evaluator was created.
- description str
- The description of the evaluator.
- evaluator_
arn str - The Amazon Resource Name (ARN) of the evaluator.
- evaluator_
config EvaluatorConfig - The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
- evaluator_
id str - The unique identifier of the evaluator.
- evaluator_
type EvaluatorType - The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
- kms_
key_ strarn - The ARN of the KMS key used to encrypt evaluator data.
- level
Evaluator
Level - The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
- provider
Evaluator
Provider - Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
- status
Evaluator
Status - The current status of the evaluator.
- Sequence[root_Tag]
- A list of tags to assign to the evaluator.
- updated_
at str - The timestamp when the evaluator was last updated.
- created
At String - The timestamp when the evaluator was created.
- description String
- The description of the evaluator.
- evaluator
Arn String - The Amazon Resource Name (ARN) of the evaluator.
- evaluator
Config Property Map - The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
- evaluator
Id String - The unique identifier of the evaluator.
- evaluator
Type "Builtin" | "ThirdParty" | "Custom" | "Custom Code" | "Custom Derived" - The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
- kms
Key StringArn - The ARN of the KMS key used to encrypt evaluator data.
- level "TOOL_CALL" | "TRACE" | "SESSION"
- The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
- provider
"AWS" | "Deep
Eval" | "Auto Eval" | "Custom" - Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
- status "ACTIVE" | "CREATING" | "CREATE_FAILED" | "UPDATING" | "UPDATE_FAILED" | "DELETING"
- The current status of the evaluator.
- List<Property Map>
- A list of tags to assign to the evaluator.
- updated
At String - The timestamp when the evaluator was last updated.
Supporting Types
EvaluatorBedrockEvaluatorModelConfig
- Model
Id string - The identifier of the Amazon Bedrock model to use for evaluation.
- Additional
Model objectRequest Fields - Additional model-specific request fields.
- Inference
Config Pulumi.Aws Native. Bedrock Agent Core. Inputs. Evaluator Inference Configuration
- Model
Id string - The identifier of the Amazon Bedrock model to use for evaluation.
- Additional
Model interface{}Request Fields - Additional model-specific request fields.
- Inference
Config EvaluatorInference Configuration
- model_
id string - The identifier of the Amazon Bedrock model to use for evaluation.
- additional_
model_ anyrequest_ fields - Additional model-specific request fields.
- inference_
config object
- model
Id String - The identifier of the Amazon Bedrock model to use for evaluation.
- additional
Model ObjectRequest Fields - Additional model-specific request fields.
- inference
Config EvaluatorInference Configuration
- model
Id string - The identifier of the Amazon Bedrock model to use for evaluation.
- additional
Model anyRequest Fields - Additional model-specific request fields.
- inference
Config EvaluatorInference Configuration
- model_
id str - The identifier of the Amazon Bedrock model to use for evaluation.
- additional_
model_ Anyrequest_ fields - Additional model-specific request fields.
- inference_
config EvaluatorInference Configuration
- model
Id String - The identifier of the Amazon Bedrock model to use for evaluation.
- additional
Model AnyRequest Fields - Additional model-specific request fields.
- inference
Config Property Map
EvaluatorCategoricalScaleDefinition
- Definition string
- The description that explains what this categorical rating represents.
- Label string
- The label of this categorical rating option.
- Definition string
- The description that explains what this categorical rating represents.
- Label string
- The label of this categorical rating option.
- definition string
- The description that explains what this categorical rating represents.
- label string
- The label of this categorical rating option.
- definition String
- The description that explains what this categorical rating represents.
- label String
- The label of this categorical rating option.
- definition string
- The description that explains what this categorical rating represents.
- label string
- The label of this categorical rating option.
- definition str
- The description that explains what this categorical rating represents.
- label str
- The label of this categorical rating option.
- definition String
- The description that explains what this categorical rating represents.
- label String
- The label of this categorical rating option.
EvaluatorCodeBasedEvaluatorConfig
EvaluatorConfig
EvaluatorDerivedEvaluatorConfig
- Base
Evaluator stringId - The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
- Model
Config Pulumi.Aws Native. Bedrock Agent Core. Inputs. Evaluator Model Config
- Base
Evaluator stringId - The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
- Model
Config EvaluatorModel Config
- base_
evaluator_ stringid - The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
- model_
config object
- base
Evaluator StringId - The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
- model
Config EvaluatorModel Config
- base
Evaluator stringId - The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
- model
Config EvaluatorModel Config
- base_
evaluator_ strid - The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
- model_
config EvaluatorModel Config
- base
Evaluator StringId - The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
- model
Config Property Map
EvaluatorDimensionRatingScale
- Categorical
List<Pulumi.
Aws Native. Bedrock Agent Core. Inputs. Evaluator Categorical Scale Definition> - The categorical rating scale for this dimension.
- Numerical
List<Pulumi.
Aws Native. Bedrock Agent Core. Inputs. Evaluator Numerical Scale Definition> - The numerical rating scale for this dimension.
- Categorical
[]Evaluator
Categorical Scale Definition - The categorical rating scale for this dimension.
- Numerical
[]Evaluator
Numerical Scale Definition - The numerical rating scale for this dimension.
- categorical list(object)
- The categorical rating scale for this dimension.
- numerical list(object)
- The numerical rating scale for this dimension.
- categorical
List<Evaluator
Categorical Scale Definition> - The categorical rating scale for this dimension.
- numerical
List<Evaluator
Numerical Scale Definition> - The numerical rating scale for this dimension.
- categorical
Evaluator
Categorical Scale Definition[] - The categorical rating scale for this dimension.
- numerical
Evaluator
Numerical Scale Definition[] - The numerical rating scale for this dimension.
- categorical
Sequence[Evaluator
Categorical Scale Definition] - The categorical rating scale for this dimension.
- numerical
Sequence[Evaluator
Numerical Scale Definition] - The numerical rating scale for this dimension.
- categorical List<Property Map>
- The categorical rating scale for this dimension.
- numerical List<Property Map>
- The numerical rating scale for this dimension.
EvaluatorEvaluationDimension
- Dimension
Instruction string - The instruction to the evaluator model describing what to judge for this dimension.
- Dimension
Name string - The name of the dimension. Must be unique within the evaluator.
- Dimension
Rating Pulumi.Scale Aws Native. Bedrock Agent Core. Inputs. Evaluator Dimension Rating Scale
- Dimension
Instruction string - The instruction to the evaluator model describing what to judge for this dimension.
- Dimension
Name string - The name of the dimension. Must be unique within the evaluator.
- Dimension
Rating EvaluatorScale Dimension Rating Scale
- dimension_
instruction string - The instruction to the evaluator model describing what to judge for this dimension.
- dimension_
name string - The name of the dimension. Must be unique within the evaluator.
- dimension_
rating_ objectscale
- dimension
Instruction String - The instruction to the evaluator model describing what to judge for this dimension.
- dimension
Name String - The name of the dimension. Must be unique within the evaluator.
- dimension
Rating EvaluatorScale Dimension Rating Scale
- dimension
Instruction string - The instruction to the evaluator model describing what to judge for this dimension.
- dimension
Name string - The name of the dimension. Must be unique within the evaluator.
- dimension
Rating EvaluatorScale Dimension Rating Scale
- dimension_
instruction str - The instruction to the evaluator model describing what to judge for this dimension.
- dimension_
name str - The name of the dimension. Must be unique within the evaluator.
- dimension_
rating_ Evaluatorscale Dimension Rating Scale
- dimension
Instruction String - The instruction to the evaluator model describing what to judge for this dimension.
- dimension
Name String - The name of the dimension. Must be unique within the evaluator.
- dimension
Rating Property MapScale
EvaluatorInferenceConfiguration
- Max
Tokens int - The maximum number of tokens to generate in the model response.
- Stop
Sequences List<string> - The sequences that make the model stop generating further tokens.
- Temperature double
- The temperature value that controls randomness in the model's responses.
- Top
P double - The top-p sampling parameter that controls the diversity of the model's responses.
- Max
Tokens int - The maximum number of tokens to generate in the model response.
- Stop
Sequences []string - The sequences that make the model stop generating further tokens.
- Temperature float64
- The temperature value that controls randomness in the model's responses.
- Top
P float64 - The top-p sampling parameter that controls the diversity of the model's responses.
- max_
tokens number - The maximum number of tokens to generate in the model response.
- stop_
sequences list(string) - The sequences that make the model stop generating further tokens.
- temperature number
- The temperature value that controls randomness in the model's responses.
- top_
p number - The top-p sampling parameter that controls the diversity of the model's responses.
- max
Tokens Integer - The maximum number of tokens to generate in the model response.
- stop
Sequences List<String> - The sequences that make the model stop generating further tokens.
- temperature Double
- The temperature value that controls randomness in the model's responses.
- top
P Double - The top-p sampling parameter that controls the diversity of the model's responses.
- max
Tokens number - The maximum number of tokens to generate in the model response.
- stop
Sequences string[] - The sequences that make the model stop generating further tokens.
- temperature number
- The temperature value that controls randomness in the model's responses.
- top
P number - The top-p sampling parameter that controls the diversity of the model's responses.
- max_
tokens int - The maximum number of tokens to generate in the model response.
- stop_
sequences Sequence[str] - The sequences that make the model stop generating further tokens.
- temperature float
- The temperature value that controls randomness in the model's responses.
- top_
p float - The top-p sampling parameter that controls the diversity of the model's responses.
- max
Tokens Number - The maximum number of tokens to generate in the model response.
- stop
Sequences List<String> - The sequences that make the model stop generating further tokens.
- temperature Number
- The temperature value that controls randomness in the model's responses.
- top
P Number - The top-p sampling parameter that controls the diversity of the model's responses.
EvaluatorLambdaEvaluatorConfig
- Lambda
Arn string - The ARN of the Lambda function used for evaluation.
- Lambda
Timeout intIn Seconds - The timeout in seconds for the Lambda function invocation.
- Lambda
Arn string - The ARN of the Lambda function used for evaluation.
- Lambda
Timeout intIn Seconds - The timeout in seconds for the Lambda function invocation.
- lambda_
arn string - The ARN of the Lambda function used for evaluation.
- lambda_
timeout_ numberin_ seconds - The timeout in seconds for the Lambda function invocation.
- lambda
Arn String - The ARN of the Lambda function used for evaluation.
- lambda
Timeout IntegerIn Seconds - The timeout in seconds for the Lambda function invocation.
- lambda
Arn string - The ARN of the Lambda function used for evaluation.
- lambda
Timeout numberIn Seconds - The timeout in seconds for the Lambda function invocation.
- lambda_
arn str - The ARN of the Lambda function used for evaluation.
- lambda_
timeout_ intin_ seconds - The timeout in seconds for the Lambda function invocation.
- lambda
Arn String - The ARN of the Lambda function used for evaluation.
- lambda
Timeout NumberIn Seconds - The timeout in seconds for the Lambda function invocation.
EvaluatorLevel
| Name | Value |
|---|---|
| Tool | TOOL_CALL |
| Trace | TRACE |
| Session | SESSION |
| Name | Value |
|---|---|
| Evaluator | TOOL_CALL |
| Evaluator | TRACE |
| Evaluator | SESSION |
| Value |
|---|
TOOL_CALL |
TRACE |
SESSION |
| Name | Value |
|---|---|
| Tool | TOOL_CALL |
| Trace | TRACE |
| Session | SESSION |
| Name | Value |
|---|---|
| Tool | TOOL_CALL |
| Trace | TRACE |
| Session | SESSION |
| Name | Value |
|---|---|
| TOOL_CALL | TOOL_CALL |
| TRACE | TRACE |
| SESSION | SESSION |
| Value |
|---|
TOOL_CALL |
TRACE |
SESSION |
EvaluatorLlmAsAJudgeEvaluatorConfig
- Instructions string
- The evaluation instructions that guide the language model in assessing agent performance.
- Model
Config Pulumi.Aws Native. Bedrock Agent Core. Inputs. Evaluator Model Config - Rating
Scale Pulumi.Aws Native. Bedrock Agent Core. Inputs. Evaluator Rating Scale
- Instructions string
- The evaluation instructions that guide the language model in assessing agent performance.
- Model
Config EvaluatorModel Config - Rating
Scale EvaluatorRating Scale
- instructions string
- The evaluation instructions that guide the language model in assessing agent performance.
- model_
config object - rating_
scale object
- instructions String
- The evaluation instructions that guide the language model in assessing agent performance.
- model
Config EvaluatorModel Config - rating
Scale EvaluatorRating Scale
- instructions string
- The evaluation instructions that guide the language model in assessing agent performance.
- model
Config EvaluatorModel Config - rating
Scale EvaluatorRating Scale
- instructions str
- The evaluation instructions that guide the language model in assessing agent performance.
- model_
config EvaluatorModel Config - rating_
scale EvaluatorRating Scale
- instructions String
- The evaluation instructions that guide the language model in assessing agent performance.
- model
Config Property Map - rating
Scale Property Map
EvaluatorModelConfig
EvaluatorNumericalScaleDefinition
- Definition string
- The description that explains what this numerical rating represents.
- Label string
- The label that describes this numerical rating option.
- Value double
- The numerical value for this rating scale option.
- Definition string
- The description that explains what this numerical rating represents.
- Label string
- The label that describes this numerical rating option.
- Value float64
- The numerical value for this rating scale option.
- definition string
- The description that explains what this numerical rating represents.
- label string
- The label that describes this numerical rating option.
- value number
- The numerical value for this rating scale option.
- definition String
- The description that explains what this numerical rating represents.
- label String
- The label that describes this numerical rating option.
- value Double
- The numerical value for this rating scale option.
- definition string
- The description that explains what this numerical rating represents.
- label string
- The label that describes this numerical rating option.
- value number
- The numerical value for this rating scale option.
- definition str
- The description that explains what this numerical rating represents.
- label str
- The label that describes this numerical rating option.
- value float
- The numerical value for this rating scale option.
- definition String
- The description that explains what this numerical rating represents.
- label String
- The label that describes this numerical rating option.
- value Number
- The numerical value for this rating scale option.
EvaluatorOpenResponsesEvaluatorModelConfig
- Model
Id string - The identifier of the model to use for evaluation.
- Max
Output intTokens - The maximum number of output tokens to generate, including visible output and reasoning tokens.
- Reasoning
Pulumi.
Aws Native. Bedrock Agent Core. Inputs. Evaluator Reasoning Configuration - Temperature double
- The sampling temperature between 0 and 2.
- Top
P double - The nucleus sampling probability mass between 0 and 1.
- Model
Id string - The identifier of the model to use for evaluation.
- Max
Output intTokens - The maximum number of output tokens to generate, including visible output and reasoning tokens.
- Reasoning
Evaluator
Reasoning Configuration - Temperature float64
- The sampling temperature between 0 and 2.
- Top
P float64 - The nucleus sampling probability mass between 0 and 1.
- model_
id string - The identifier of the model to use for evaluation.
- max_
output_ numbertokens - The maximum number of output tokens to generate, including visible output and reasoning tokens.
- reasoning object
- temperature number
- The sampling temperature between 0 and 2.
- top_
p number - The nucleus sampling probability mass between 0 and 1.
- model
Id String - The identifier of the model to use for evaluation.
- max
Output IntegerTokens - The maximum number of output tokens to generate, including visible output and reasoning tokens.
- reasoning
Evaluator
Reasoning Configuration - temperature Double
- The sampling temperature between 0 and 2.
- top
P Double - The nucleus sampling probability mass between 0 and 1.
- model
Id string - The identifier of the model to use for evaluation.
- max
Output numberTokens - The maximum number of output tokens to generate, including visible output and reasoning tokens.
- reasoning
Evaluator
Reasoning Configuration - temperature number
- The sampling temperature between 0 and 2.
- top
P number - The nucleus sampling probability mass between 0 and 1.
- model_
id str - The identifier of the model to use for evaluation.
- max_
output_ inttokens - The maximum number of output tokens to generate, including visible output and reasoning tokens.
- reasoning
Evaluator
Reasoning Configuration - temperature float
- The sampling temperature between 0 and 2.
- top_
p float - The nucleus sampling probability mass between 0 and 1.
- model
Id String - The identifier of the model to use for evaluation.
- max
Output NumberTokens - The maximum number of output tokens to generate, including visible output and reasoning tokens.
- reasoning Property Map
- temperature Number
- The sampling temperature between 0 and 2.
- top
P Number - The nucleus sampling probability mass between 0 and 1.
EvaluatorProvider
| Name | Value |
|---|---|
| Aws | AWS |
| Deep | DeepEval |
| Auto | AutoEval |
| Custom | Custom |
| Name | Value |
|---|---|
| Evaluator | AWS |
| Evaluator | DeepEval |
| Evaluator | AutoEval |
| Evaluator | Custom |
| Value |
|---|
AWS |
DeepEval |
AutoEval |
Custom |
| Name | Value |
|---|---|
| Aws | AWS |
| Deep | DeepEval |
| Auto | AutoEval |
| Custom | Custom |
| Name | Value |
|---|---|
| Aws | AWS |
| Deep | DeepEval |
| Auto | AutoEval |
| Custom | Custom |
| Name | Value |
|---|---|
| AWS | AWS |
| DEEP_EVAL | DeepEval |
| AUTO_EVAL | AutoEval |
| CUSTOM | Custom |
| Value |
|---|
AWS |
DeepEval |
AutoEval |
Custom |
EvaluatorRatingScale
- Categorical
List<Pulumi.
Aws Native. Bedrock Agent Core. Inputs. Evaluator Categorical Scale Definition> - Multi
Dimensional List<Pulumi.Aws Native. Bedrock Agent Core. Inputs. Evaluator Evaluation Dimension> - The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
- Numerical
List<Pulumi.
Aws Native. Bedrock Agent Core. Inputs. Evaluator Numerical Scale Definition>
- Categorical
[]Evaluator
Categorical Scale Definition - Multi
Dimensional []EvaluatorEvaluation Dimension - The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
- Numerical
[]Evaluator
Numerical Scale Definition
- categorical list(object)
- multi_
dimensional list(object) - The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
- numerical list(object)
- categorical
List<Evaluator
Categorical Scale Definition> - multi
Dimensional List<EvaluatorEvaluation Dimension> - The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
- numerical
List<Evaluator
Numerical Scale Definition>
- categorical
Evaluator
Categorical Scale Definition[] - multi
Dimensional EvaluatorEvaluation Dimension[] - The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
- numerical
Evaluator
Numerical Scale Definition[]
- categorical
Sequence[Evaluator
Categorical Scale Definition] - multi_
dimensional Sequence[EvaluatorEvaluation Dimension] - The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
- numerical
Sequence[Evaluator
Numerical Scale Definition]
- categorical List<Property Map>
- multi
Dimensional List<Property Map> - The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
- numerical List<Property Map>
EvaluatorReasoningConfiguration
- Effort string
- The level of reasoning effort the model applies.
- Effort string
- The level of reasoning effort the model applies.
- effort string
- The level of reasoning effort the model applies.
- effort String
- The level of reasoning effort the model applies.
- effort string
- The level of reasoning effort the model applies.
- effort str
- The level of reasoning effort the model applies.
- effort String
- The level of reasoning effort the model applies.
EvaluatorStatus
| Name | Value |
|---|---|
| Active | ACTIVE |
| Creating | CREATING |
| Create | CREATE_FAILED |
| Updating | UPDATING |
| Update | UPDATE_FAILED |
| Deleting | DELETING |
| Name | Value |
|---|---|
| Evaluator | ACTIVE |
| Evaluator | CREATING |
| Evaluator | CREATE_FAILED |
| Evaluator | UPDATING |
| Evaluator | UPDATE_FAILED |
| Evaluator | DELETING |
| Value |
|---|
ACTIVE |
CREATING |
CREATE_FAILED |
UPDATING |
UPDATE_FAILED |
DELETING |
| Name | Value |
|---|---|
| Active | ACTIVE |
| Creating | CREATING |
| Create | CREATE_FAILED |
| Updating | UPDATING |
| Update | UPDATE_FAILED |
| Deleting | DELETING |
| Name | Value |
|---|---|
| Active | ACTIVE |
| Creating | CREATING |
| Create | CREATE_FAILED |
| Updating | UPDATING |
| Update | UPDATE_FAILED |
| Deleting | DELETING |
| Name | Value |
|---|---|
| ACTIVE | ACTIVE |
| CREATING | CREATING |
| CREATE_FAILED | CREATE_FAILED |
| UPDATING | UPDATING |
| UPDATE_FAILED | UPDATE_FAILED |
| DELETING | DELETING |
| Value |
|---|
ACTIVE |
CREATING |
CREATE_FAILED |
UPDATING |
UPDATE_FAILED |
DELETING |
EvaluatorType
| Name | Value |
|---|---|
| Builtin | Builtin |
| Third | ThirdParty |
| Custom | Custom |
| Custom | CustomCode |
| Custom | CustomDerived |
| Name | Value |
|---|---|
| Evaluator | Builtin |
| Evaluator | ThirdParty |
| Evaluator | Custom |
| Evaluator | CustomCode |
| Evaluator | CustomDerived |
| Value |
|---|
Builtin |
ThirdParty |
Custom |
CustomCode |
CustomDerived |
| Name | Value |
|---|---|
| Builtin | Builtin |
| Third | ThirdParty |
| Custom | Custom |
| Custom | CustomCode |
| Custom | CustomDerived |
| Name | Value |
|---|---|
| Builtin | Builtin |
| Third | ThirdParty |
| Custom | Custom |
| Custom | CustomCode |
| Custom | CustomDerived |
| Name | Value |
|---|---|
| BUILTIN | Builtin |
| THIRD_PARTY | ThirdParty |
| CUSTOM | Custom |
| CUSTOM_CODE | CustomCode |
| CUSTOM_DERIVED | CustomDerived |
| Value |
|---|
Builtin |
ThirdParty |
Custom |
CustomCode |
CustomDerived |
Tag
Package Details
- Repository
- AWS Native pulumi/pulumi-aws-native
- License
- Apache-2.0
We recommend new projects start with resources from the AWS provider.
published on Wednesday, Nov 18, 2026 by Pulumi