1. Registry
  2. Packages
  3. AWS Cloud Control
  4. API Docs
  5. bedrockagentcore
  6. Evaluator

We recommend new projects start with resources from the AWS provider.

Viewing docs for AWS Cloud Control v1.86.0
published on Wednesday, Nov 18, 2026 by Pulumi
aws-native logo aws-native logo AWS Cloud Control

We recommend new projects start with resources from the AWS provider.

Viewing docs for AWS Cloud Control v1.86.0
published on Wednesday, Nov 18, 2026 by Pulumi

    Resource Type definition for AWS::BedrockAgentCore::Evaluator - Creates a custom evaluator for agent quality assessment using LLM-as-a-Judge, code-based, or derived evaluator configurations. Derived evaluators run an existing built-in or third-party evaluator’s logic on a model that you supply.

    Create Evaluator Resource

    Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

    Constructor syntax

    new Evaluator(name: string, args: EvaluatorArgs, opts?: CustomResourceOptions);
    @overload
    def Evaluator(resource_name: str,
                  args: EvaluatorArgs,
                  opts: Optional[ResourceOptions] = None)
    
    @overload
    def Evaluator(resource_name: str,
                  opts: Optional[ResourceOptions] = None,
                  evaluator_config: Optional[EvaluatorConfigArgs] = None,
                  level: Optional[EvaluatorLevel] = None,
                  description: Optional[str] = None,
                  evaluator_name: Optional[str] = None,
                  kms_key_arn: Optional[str] = None,
                  tags: Optional[Sequence[_root_inputs.TagArgs]] = None)
    func NewEvaluator(ctx *Context, name string, args EvaluatorArgs, opts ...ResourceOption) (*Evaluator, error)
    public Evaluator(string name, EvaluatorArgs args, CustomResourceOptions? opts = null)
    public Evaluator(String name, EvaluatorArgs args)
    public Evaluator(String name, EvaluatorArgs args, CustomResourceOptions options)
    
    type: aws-native:bedrockagentcore:Evaluator
    properties: # The arguments to resource properties.
    options: # Bag of options to control resource's behavior.
    
    
    resource "aws-native_bedrockagentcore_evaluator" "name" {
        # resource properties
    }

    Parameters

    name string
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    opts CustomResourceOptions
    Bag of options to control resource's behavior.
    resource_name str
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    opts ResourceOptions
    Bag of options to control resource's behavior.
    ctx Context
    Context object for the current deployment.
    name string
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    opts ResourceOption
    Bag of options to control resource's behavior.
    name string
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    opts CustomResourceOptions
    Bag of options to control resource's behavior.
    name String
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    options CustomResourceOptions
    Bag of options to control resource's behavior.

    Evaluator Resource Properties

    To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

    Inputs

    In Python, inputs that are objects can be passed either as argument classes or as dictionary literals.

    The Evaluator resource accepts the following input properties:

    EvaluatorConfig Pulumi.AwsNative.BedrockAgentCore.Inputs.EvaluatorConfig
    The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
    Level Pulumi.AwsNative.BedrockAgentCore.EvaluatorLevel
    The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
    Description string
    The description of the evaluator.
    EvaluatorName string
    The name of the evaluator. Must be unique within your account.
    KmsKeyArn string
    The ARN of the KMS key used to encrypt evaluator data.
    Tags List<Pulumi.AwsNative.Inputs.Tag>
    A list of tags to assign to the evaluator.
    EvaluatorConfig EvaluatorConfigArgs
    The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
    Level EvaluatorLevel
    The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
    Description string
    The description of the evaluator.
    EvaluatorName string
    The name of the evaluator. Must be unique within your account.
    KmsKeyArn string
    The ARN of the KMS key used to encrypt evaluator data.
    Tags TagArgs
    A list of tags to assign to the evaluator.
    evaluator_config object
    The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
    level "TOOL_CALL" | "TRACE" | "SESSION"
    The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
    description string
    The description of the evaluator.
    evaluator_name string
    The name of the evaluator. Must be unique within your account.
    kms_key_arn string
    The ARN of the KMS key used to encrypt evaluator data.
    tags list(object)
    A list of tags to assign to the evaluator.
    evaluatorConfig EvaluatorConfig
    The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
    level EvaluatorLevel
    The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
    description String
    The description of the evaluator.
    evaluatorName String
    The name of the evaluator. Must be unique within your account.
    kmsKeyArn String
    The ARN of the KMS key used to encrypt evaluator data.
    tags List<Tag>
    A list of tags to assign to the evaluator.
    evaluatorConfig EvaluatorConfig
    The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
    level EvaluatorLevel
    The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
    description string
    The description of the evaluator.
    evaluatorName string
    The name of the evaluator. Must be unique within your account.
    kmsKeyArn string
    The ARN of the KMS key used to encrypt evaluator data.
    tags Tag[]
    A list of tags to assign to the evaluator.
    evaluator_config EvaluatorConfigArgs
    The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
    level EvaluatorLevel
    The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
    description str
    The description of the evaluator.
    evaluator_name str
    The name of the evaluator. Must be unique within your account.
    kms_key_arn str
    The ARN of the KMS key used to encrypt evaluator data.
    tags Sequence[TagArgs]
    A list of tags to assign to the evaluator.
    evaluatorConfig Property Map
    The configuration for the evaluator, including LLM-as-a-Judge, code-based, or derived settings.
    level "TOOL_CALL" | "TRACE" | "SESSION"
    The evaluation level that determines the scope of evaluation. Required for all evaluator configuration types. For Derived evaluators it must match the level of the base evaluator identified by BaseEvaluatorId.
    description String
    The description of the evaluator.
    evaluatorName String
    The name of the evaluator. Must be unique within your account.
    kmsKeyArn String
    The ARN of the KMS key used to encrypt evaluator data.
    tags List<Property Map>
    A list of tags to assign to the evaluator.

    Outputs

    All input properties are implicitly available as output properties. Additionally, the Evaluator resource produces the following output properties:

    CreatedAt string
    The timestamp when the evaluator was created.
    EvaluatorArn string
    The Amazon Resource Name (ARN) of the evaluator.
    EvaluatorId string
    The unique identifier of the evaluator.
    EvaluatorType Pulumi.AwsNative.BedrockAgentCore.EvaluatorType
    The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
    Id string
    The provider-assigned unique ID for this managed resource.
    Provider Pulumi.AwsNative.BedrockAgentCore.EvaluatorProvider
    Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
    Status Pulumi.AwsNative.BedrockAgentCore.EvaluatorStatus
    The current status of the evaluator.
    UpdatedAt string
    The timestamp when the evaluator was last updated.
    CreatedAt string
    The timestamp when the evaluator was created.
    EvaluatorArn string
    The Amazon Resource Name (ARN) of the evaluator.
    EvaluatorId string
    The unique identifier of the evaluator.
    EvaluatorType EvaluatorType
    The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
    Id string
    The provider-assigned unique ID for this managed resource.
    Provider EvaluatorProvider
    Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
    Status EvaluatorStatus
    The current status of the evaluator.
    UpdatedAt string
    The timestamp when the evaluator was last updated.
    created_at string
    The timestamp when the evaluator was created.
    evaluator_arn string
    The Amazon Resource Name (ARN) of the evaluator.
    evaluator_id string
    The unique identifier of the evaluator.
    evaluator_type "Builtin" | "ThirdParty" | "Custom" | "CustomCode" | "CustomDerived"
    The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
    id string
    The provider-assigned unique ID for this managed resource.
    provider "AWS" | "DeepEval" | "AutoEval" | "Custom"
    Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
    status "ACTIVE" | "CREATING" | "CREATE_FAILED" | "UPDATING" | "UPDATE_FAILED" | "DELETING"
    The current status of the evaluator.
    updated_at string
    The timestamp when the evaluator was last updated.
    createdAt String
    The timestamp when the evaluator was created.
    evaluatorArn String
    The Amazon Resource Name (ARN) of the evaluator.
    evaluatorId String
    The unique identifier of the evaluator.
    evaluatorType EvaluatorType
    The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
    id String
    The provider-assigned unique ID for this managed resource.
    provider EvaluatorProvider
    Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
    status EvaluatorStatus
    The current status of the evaluator.
    updatedAt String
    The timestamp when the evaluator was last updated.
    createdAt string
    The timestamp when the evaluator was created.
    evaluatorArn string
    The Amazon Resource Name (ARN) of the evaluator.
    evaluatorId string
    The unique identifier of the evaluator.
    evaluatorType EvaluatorType
    The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
    id string
    The provider-assigned unique ID for this managed resource.
    provider EvaluatorProvider
    Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
    status EvaluatorStatus
    The current status of the evaluator.
    updatedAt string
    The timestamp when the evaluator was last updated.
    created_at str
    The timestamp when the evaluator was created.
    evaluator_arn str
    The Amazon Resource Name (ARN) of the evaluator.
    evaluator_id str
    The unique identifier of the evaluator.
    evaluator_type EvaluatorType
    The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
    id str
    The provider-assigned unique ID for this managed resource.
    provider EvaluatorProvider
    Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
    status EvaluatorStatus
    The current status of the evaluator.
    updated_at str
    The timestamp when the evaluator was last updated.
    createdAt String
    The timestamp when the evaluator was created.
    evaluatorArn String
    The Amazon Resource Name (ARN) of the evaluator.
    evaluatorId String
    The unique identifier of the evaluator.
    evaluatorType "Builtin" | "ThirdParty" | "Custom" | "CustomCode" | "CustomDerived"
    The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.
    id String
    The provider-assigned unique ID for this managed resource.
    provider "AWS" | "DeepEval" | "AutoEval" | "Custom"
    Where the evaluator's logic comes from: AWS, a third-party library (DeepEval or AutoEval), or you. Derived evaluators inherit the provider of their base evaluator.
    status "ACTIVE" | "CREATING" | "CREATE_FAILED" | "UPDATING" | "UPDATE_FAILED" | "DELETING"
    The current status of the evaluator.
    updatedAt String
    The timestamp when the evaluator was last updated.

    Look up Existing Evaluator Resource

    Get an existing Evaluator resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

    public static get(name: string, id: Input<ID>, opts?: CustomResourceOptions): Evaluator
    @staticmethod
    def get(resource_name: str,
            id: str,
            opts: Optional[ResourceOptions] = None) -> Evaluator
    func GetEvaluator(ctx *Context, name string, id IDInput, state *EvaluatorState, opts ...ResourceOption) (*Evaluator, error)
    public static Evaluator Get(string name, Input<string> id, CustomResourceOptions? opts = null)
    public static Evaluator get(String name, Output<String> id, CustomResourceOptions options)
    resources:  existingEvaluator:    type: aws-native:bedrockagentcore:Evaluator    get:      id: ${id}
    import {
      to = aws-native_bedrockagentcore_evaluator.example
      id = "${id}"
    }
    
    name string
    The unique name of the resulting resource.
    id Input<ID>
    The unique provider ID of the resource to look up.
    opts CustomResourceOptions
    Bag of options to control resource's behavior.
    resource_name str
    The unique name of the resulting resource.
    id str
    The unique provider ID of the resource to look up.
    opts ResourceOptions
    Bag of options to control resource's behavior.
    ctx Context
    Context object for the current deployment.
    name string
    The unique name of the resulting resource.
    id IDInput
    The unique provider ID of the resource to look up.
    state EvaluatorState
    Pass nil. This resource has no state arguments.
    opts ResourceOption
    Bag of options to control resource's behavior.
    name string
    The unique name of the resulting resource.
    id Input<string>
    The unique provider ID of the resource to look up.
    opts CustomResourceOptions
    Bag of options to control resource's behavior.
    name String
    The unique name of the resulting resource.
    id Output<String>
    The unique provider ID of the resource to look up.
    options CustomResourceOptions
    Bag of options to control resource's behavior.

    Supporting Types

    EvaluatorBedrockEvaluatorModelConfig, EvaluatorBedrockEvaluatorModelConfigArgs

    The configuration for using Amazon Bedrock models in evaluator assessments.

    ModelId string
    The identifier of the Amazon Bedrock model to use for evaluation.
    AdditionalModelRequestFields object
    Additional model-specific request fields.
    InferenceConfig Pulumi.AwsNative.BedrockAgentCore.Inputs.EvaluatorInferenceConfiguration
    ModelId string
    The identifier of the Amazon Bedrock model to use for evaluation.
    AdditionalModelRequestFields interface{}
    Additional model-specific request fields.
    InferenceConfig EvaluatorInferenceConfiguration
    model_id string
    The identifier of the Amazon Bedrock model to use for evaluation.
    additional_model_request_fields any
    Additional model-specific request fields.
    inference_config object
    modelId String
    The identifier of the Amazon Bedrock model to use for evaluation.
    additionalModelRequestFields Object
    Additional model-specific request fields.
    inferenceConfig EvaluatorInferenceConfiguration
    modelId string
    The identifier of the Amazon Bedrock model to use for evaluation.
    additionalModelRequestFields any
    Additional model-specific request fields.
    inferenceConfig EvaluatorInferenceConfiguration
    model_id str
    The identifier of the Amazon Bedrock model to use for evaluation.
    additional_model_request_fields Any
    Additional model-specific request fields.
    inference_config EvaluatorInferenceConfiguration
    modelId String
    The identifier of the Amazon Bedrock model to use for evaluation.
    additionalModelRequestFields Any
    Additional model-specific request fields.
    inferenceConfig Property Map

    EvaluatorCategoricalScaleDefinition, EvaluatorCategoricalScaleDefinitionArgs

    A categorical rating scale option.

    Definition string
    The description that explains what this categorical rating represents.
    Label string
    The label of this categorical rating option.
    Definition string
    The description that explains what this categorical rating represents.
    Label string
    The label of this categorical rating option.
    definition string
    The description that explains what this categorical rating represents.
    label string
    The label of this categorical rating option.
    definition String
    The description that explains what this categorical rating represents.
    label String
    The label of this categorical rating option.
    definition string
    The description that explains what this categorical rating represents.
    label string
    The label of this categorical rating option.
    definition str
    The description that explains what this categorical rating represents.
    label str
    The label of this categorical rating option.
    definition String
    The description that explains what this categorical rating represents.
    label String
    The label of this categorical rating option.

    EvaluatorCodeBasedEvaluatorConfig, EvaluatorCodeBasedEvaluatorConfigArgs

    The configuration for code-based evaluation using a Lambda function.

    EvaluatorConfig, EvaluatorConfigArgs

    The configuration that defines how an evaluator assesses agent performance.

    EvaluatorDerivedEvaluatorConfig, EvaluatorDerivedEvaluatorConfigArgs

    The configuration for an evaluator derived from an existing base evaluator (a built-in or third-party evaluator), run on your own model. The base evaluator supplies the prompt and scoring logic.

    BaseEvaluatorId string
    The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
    ModelConfig Pulumi.AwsNative.BedrockAgentCore.Inputs.EvaluatorModelConfig
    BaseEvaluatorId string
    The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
    ModelConfig EvaluatorModelConfig
    base_evaluator_id string
    The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
    model_config object
    baseEvaluatorId String
    The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
    modelConfig EvaluatorModelConfig
    baseEvaluatorId string
    The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
    modelConfig EvaluatorModelConfig
    base_evaluator_id str
    The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
    model_config EvaluatorModelConfig
    baseEvaluatorId String
    The identifier of the base evaluator whose logic to run (a Builtin.* or ThirdParty.* evaluator).
    modelConfig Property Map

    EvaluatorDimensionRatingScale, EvaluatorDimensionRatingScaleArgs

    The rating scale for one dimension of an LLM-as-a-Judge evaluator.

    Categorical []EvaluatorCategoricalScaleDefinition
    The categorical rating scale for this dimension.
    Numerical []EvaluatorNumericalScaleDefinition
    The numerical rating scale for this dimension.
    categorical list(object)
    The categorical rating scale for this dimension.
    numerical list(object)
    The numerical rating scale for this dimension.
    categorical List<EvaluatorCategoricalScaleDefinition>
    The categorical rating scale for this dimension.
    numerical List<EvaluatorNumericalScaleDefinition>
    The numerical rating scale for this dimension.
    categorical EvaluatorCategoricalScaleDefinition[]
    The categorical rating scale for this dimension.
    numerical EvaluatorNumericalScaleDefinition[]
    The numerical rating scale for this dimension.
    categorical Sequence[EvaluatorCategoricalScaleDefinition]
    The categorical rating scale for this dimension.
    numerical Sequence[EvaluatorNumericalScaleDefinition]
    The numerical rating scale for this dimension.
    categorical List<Property Map>
    The categorical rating scale for this dimension.
    numerical List<Property Map>
    The numerical rating scale for this dimension.

    EvaluatorEvaluationDimension, EvaluatorEvaluationDimensionArgs

    One dimension of an LLM-as-a-Judge evaluator.

    DimensionInstruction string
    The instruction to the evaluator model describing what to judge for this dimension.
    DimensionName string
    The name of the dimension. Must be unique within the evaluator.
    DimensionRatingScale Pulumi.AwsNative.BedrockAgentCore.Inputs.EvaluatorDimensionRatingScale
    DimensionInstruction string
    The instruction to the evaluator model describing what to judge for this dimension.
    DimensionName string
    The name of the dimension. Must be unique within the evaluator.
    DimensionRatingScale EvaluatorDimensionRatingScale
    dimension_instruction string
    The instruction to the evaluator model describing what to judge for this dimension.
    dimension_name string
    The name of the dimension. Must be unique within the evaluator.
    dimension_rating_scale object
    dimensionInstruction String
    The instruction to the evaluator model describing what to judge for this dimension.
    dimensionName String
    The name of the dimension. Must be unique within the evaluator.
    dimensionRatingScale EvaluatorDimensionRatingScale
    dimensionInstruction string
    The instruction to the evaluator model describing what to judge for this dimension.
    dimensionName string
    The name of the dimension. Must be unique within the evaluator.
    dimensionRatingScale EvaluatorDimensionRatingScale
    dimension_instruction str
    The instruction to the evaluator model describing what to judge for this dimension.
    dimension_name str
    The name of the dimension. Must be unique within the evaluator.
    dimension_rating_scale EvaluatorDimensionRatingScale
    dimensionInstruction String
    The instruction to the evaluator model describing what to judge for this dimension.
    dimensionName String
    The name of the dimension. Must be unique within the evaluator.
    dimensionRatingScale Property Map

    EvaluatorInferenceConfiguration, EvaluatorInferenceConfigurationArgs

    The inference configuration parameters that control model behavior during evaluation.

    MaxTokens int
    The maximum number of tokens to generate in the model response.
    StopSequences List<string>
    The sequences that make the model stop generating further tokens.
    Temperature double
    The temperature value that controls randomness in the model's responses.
    TopP double
    The top-p sampling parameter that controls the diversity of the model's responses.
    MaxTokens int
    The maximum number of tokens to generate in the model response.
    StopSequences []string
    The sequences that make the model stop generating further tokens.
    Temperature float64
    The temperature value that controls randomness in the model's responses.
    TopP float64
    The top-p sampling parameter that controls the diversity of the model's responses.
    max_tokens number
    The maximum number of tokens to generate in the model response.
    stop_sequences list(string)
    The sequences that make the model stop generating further tokens.
    temperature number
    The temperature value that controls randomness in the model's responses.
    top_p number
    The top-p sampling parameter that controls the diversity of the model's responses.
    maxTokens Integer
    The maximum number of tokens to generate in the model response.
    stopSequences List<String>
    The sequences that make the model stop generating further tokens.
    temperature Double
    The temperature value that controls randomness in the model's responses.
    topP Double
    The top-p sampling parameter that controls the diversity of the model's responses.
    maxTokens number
    The maximum number of tokens to generate in the model response.
    stopSequences string[]
    The sequences that make the model stop generating further tokens.
    temperature number
    The temperature value that controls randomness in the model's responses.
    topP number
    The top-p sampling parameter that controls the diversity of the model's responses.
    max_tokens int
    The maximum number of tokens to generate in the model response.
    stop_sequences Sequence[str]
    The sequences that make the model stop generating further tokens.
    temperature float
    The temperature value that controls randomness in the model's responses.
    top_p float
    The top-p sampling parameter that controls the diversity of the model's responses.
    maxTokens Number
    The maximum number of tokens to generate in the model response.
    stopSequences List<String>
    The sequences that make the model stop generating further tokens.
    temperature Number
    The temperature value that controls randomness in the model's responses.
    topP Number
    The top-p sampling parameter that controls the diversity of the model's responses.

    EvaluatorLambdaEvaluatorConfig, EvaluatorLambdaEvaluatorConfigArgs

    The Lambda function configuration for code-based evaluation.

    LambdaArn string
    The ARN of the Lambda function used for evaluation.
    LambdaTimeoutInSeconds int
    The timeout in seconds for the Lambda function invocation.
    LambdaArn string
    The ARN of the Lambda function used for evaluation.
    LambdaTimeoutInSeconds int
    The timeout in seconds for the Lambda function invocation.
    lambda_arn string
    The ARN of the Lambda function used for evaluation.
    lambda_timeout_in_seconds number
    The timeout in seconds for the Lambda function invocation.
    lambdaArn String
    The ARN of the Lambda function used for evaluation.
    lambdaTimeoutInSeconds Integer
    The timeout in seconds for the Lambda function invocation.
    lambdaArn string
    The ARN of the Lambda function used for evaluation.
    lambdaTimeoutInSeconds number
    The timeout in seconds for the Lambda function invocation.
    lambda_arn str
    The ARN of the Lambda function used for evaluation.
    lambda_timeout_in_seconds int
    The timeout in seconds for the Lambda function invocation.
    lambdaArn String
    The ARN of the Lambda function used for evaluation.
    lambdaTimeoutInSeconds Number
    The timeout in seconds for the Lambda function invocation.

    EvaluatorLevel

    NameValue
    ToolCallTOOL_CALL
    TraceTRACE
    SessionSESSION
    NameValue
    EvaluatorLevelToolCallTOOL_CALL
    EvaluatorLevelTraceTRACE
    EvaluatorLevelSessionSESSION
    Value
    TOOL_CALL
    TRACE
    SESSION
    NameValue
    ToolCallTOOL_CALL
    TraceTRACE
    SessionSESSION
    NameValue
    ToolCallTOOL_CALL
    TraceTRACE
    SessionSESSION
    NameValue
    TOOL_CALLTOOL_CALL
    TRACETRACE
    SESSIONSESSION
    Value
    TOOL_CALL
    TRACE
    SESSION

    EvaluatorLlmAsAJudgeEvaluatorConfig, EvaluatorLlmAsAJudgeEvaluatorConfigArgs

    The configuration for LLM-as-a-Judge evaluation.

    Instructions string
    The evaluation instructions that guide the language model in assessing agent performance.
    ModelConfig EvaluatorModelConfig
    RatingScale EvaluatorRatingScale
    instructions string
    The evaluation instructions that guide the language model in assessing agent performance.
    model_config object
    rating_scale object
    instructions String
    The evaluation instructions that guide the language model in assessing agent performance.
    modelConfig EvaluatorModelConfig
    ratingScale EvaluatorRatingScale
    instructions string
    The evaluation instructions that guide the language model in assessing agent performance.
    modelConfig EvaluatorModelConfig
    ratingScale EvaluatorRatingScale
    instructions str
    The evaluation instructions that guide the language model in assessing agent performance.
    model_config EvaluatorModelConfig
    rating_scale EvaluatorRatingScale
    instructions String
    The evaluation instructions that guide the language model in assessing agent performance.
    modelConfig Property Map
    ratingScale Property Map

    EvaluatorModelConfig, EvaluatorModelConfigArgs

    The model configuration that specifies which foundation model to use for evaluation.

    EvaluatorNumericalScaleDefinition, EvaluatorNumericalScaleDefinitionArgs

    A numerical rating scale option.

    Definition string
    The description that explains what this numerical rating represents.
    Label string
    The label that describes this numerical rating option.
    Value double
    The numerical value for this rating scale option.
    Definition string
    The description that explains what this numerical rating represents.
    Label string
    The label that describes this numerical rating option.
    Value float64
    The numerical value for this rating scale option.
    definition string
    The description that explains what this numerical rating represents.
    label string
    The label that describes this numerical rating option.
    value number
    The numerical value for this rating scale option.
    definition String
    The description that explains what this numerical rating represents.
    label String
    The label that describes this numerical rating option.
    value Double
    The numerical value for this rating scale option.
    definition string
    The description that explains what this numerical rating represents.
    label string
    The label that describes this numerical rating option.
    value number
    The numerical value for this rating scale option.
    definition str
    The description that explains what this numerical rating represents.
    label str
    The label that describes this numerical rating option.
    value float
    The numerical value for this rating scale option.
    definition String
    The description that explains what this numerical rating represents.
    label String
    The label that describes this numerical rating option.
    value Number
    The numerical value for this rating scale option.

    EvaluatorOpenResponsesEvaluatorModelConfig, EvaluatorOpenResponsesEvaluatorModelConfigArgs

    The configuration for using OpenResponses-compatible models in evaluator assessments.

    ModelId string
    The identifier of the model to use for evaluation.
    MaxOutputTokens int
    The maximum number of output tokens to generate, including visible output and reasoning tokens.
    Reasoning Pulumi.AwsNative.BedrockAgentCore.Inputs.EvaluatorReasoningConfiguration
    Temperature double
    The sampling temperature between 0 and 2.
    TopP double
    The nucleus sampling probability mass between 0 and 1.
    ModelId string
    The identifier of the model to use for evaluation.
    MaxOutputTokens int
    The maximum number of output tokens to generate, including visible output and reasoning tokens.
    Reasoning EvaluatorReasoningConfiguration
    Temperature float64
    The sampling temperature between 0 and 2.
    TopP float64
    The nucleus sampling probability mass between 0 and 1.
    model_id string
    The identifier of the model to use for evaluation.
    max_output_tokens number
    The maximum number of output tokens to generate, including visible output and reasoning tokens.
    reasoning object
    temperature number
    The sampling temperature between 0 and 2.
    top_p number
    The nucleus sampling probability mass between 0 and 1.
    modelId String
    The identifier of the model to use for evaluation.
    maxOutputTokens Integer
    The maximum number of output tokens to generate, including visible output and reasoning tokens.
    reasoning EvaluatorReasoningConfiguration
    temperature Double
    The sampling temperature between 0 and 2.
    topP Double
    The nucleus sampling probability mass between 0 and 1.
    modelId string
    The identifier of the model to use for evaluation.
    maxOutputTokens number
    The maximum number of output tokens to generate, including visible output and reasoning tokens.
    reasoning EvaluatorReasoningConfiguration
    temperature number
    The sampling temperature between 0 and 2.
    topP number
    The nucleus sampling probability mass between 0 and 1.
    model_id str
    The identifier of the model to use for evaluation.
    max_output_tokens int
    The maximum number of output tokens to generate, including visible output and reasoning tokens.
    reasoning EvaluatorReasoningConfiguration
    temperature float
    The sampling temperature between 0 and 2.
    top_p float
    The nucleus sampling probability mass between 0 and 1.
    modelId String
    The identifier of the model to use for evaluation.
    maxOutputTokens Number
    The maximum number of output tokens to generate, including visible output and reasoning tokens.
    reasoning Property Map
    temperature Number
    The sampling temperature between 0 and 2.
    topP Number
    The nucleus sampling probability mass between 0 and 1.

    EvaluatorProvider

    Where the evaluator's logic comes from: AWS for AWS-authored evaluators, DeepEval or AutoEval for the corresponding third-party libraries, or Custom for an evaluator that you authored.

    NameValue
    AwsAWS
    DeepEvalDeepEval
    AutoEvalAutoEval
    CustomCustom
    NameValue
    EvaluatorProviderAwsAWS
    EvaluatorProviderDeepEvalDeepEval
    EvaluatorProviderAutoEvalAutoEval
    EvaluatorProviderCustomCustom
    Value
    AWS
    DeepEval
    AutoEval
    Custom
    NameValue
    AwsAWS
    DeepEvalDeepEval
    AutoEvalAutoEval
    CustomCustom
    NameValue
    AwsAWS
    DeepEvalDeepEval
    AutoEvalAutoEval
    CustomCustom
    NameValue
    AWSAWS
    DEEP_EVALDeepEval
    AUTO_EVALAutoEval
    CUSTOMCustom
    Value
    AWS
    DeepEval
    AutoEval
    Custom

    EvaluatorRatingScale, EvaluatorRatingScaleArgs

    The rating scale that defines how evaluators should score agent performance.

    Categorical []EvaluatorCategoricalScaleDefinition
    MultiDimensional []EvaluatorEvaluationDimension
    The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
    Numerical []EvaluatorNumericalScaleDefinition
    categorical list(object)
    multi_dimensional list(object)
    The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
    numerical list(object)
    categorical List<EvaluatorCategoricalScaleDefinition>
    multiDimensional List<EvaluatorEvaluationDimension>
    The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
    numerical List<EvaluatorNumericalScaleDefinition>
    categorical EvaluatorCategoricalScaleDefinition[]
    multiDimensional EvaluatorEvaluationDimension[]
    The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
    numerical EvaluatorNumericalScaleDefinition[]
    categorical Sequence[EvaluatorCategoricalScaleDefinition]
    multi_dimensional Sequence[EvaluatorEvaluationDimension]
    The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
    numerical Sequence[EvaluatorNumericalScaleDefinition]
    categorical List<Property Map>
    multiDimensional List<Property Map>
    The dimensions an LLM-as-a-Judge evaluator scores in a single judge call. Each dimension has its own instruction and rating scale.
    numerical List<Property Map>

    EvaluatorReasoningConfiguration, EvaluatorReasoningConfigurationArgs

    The reasoning configuration for reasoning models.

    Effort string
    The level of reasoning effort the model applies.
    Effort string
    The level of reasoning effort the model applies.
    effort string
    The level of reasoning effort the model applies.
    effort String
    The level of reasoning effort the model applies.
    effort string
    The level of reasoning effort the model applies.
    effort str
    The level of reasoning effort the model applies.
    effort String
    The level of reasoning effort the model applies.

    EvaluatorStatus

    NameValue
    ActiveACTIVE
    CreatingCREATING
    CreateFailedCREATE_FAILED
    UpdatingUPDATING
    UpdateFailedUPDATE_FAILED
    DeletingDELETING
    NameValue
    EvaluatorStatusActiveACTIVE
    EvaluatorStatusCreatingCREATING
    EvaluatorStatusCreateFailedCREATE_FAILED
    EvaluatorStatusUpdatingUPDATING
    EvaluatorStatusUpdateFailedUPDATE_FAILED
    EvaluatorStatusDeletingDELETING
    Value
    ACTIVE
    CREATING
    CREATE_FAILED
    UPDATING
    UPDATE_FAILED
    DELETING
    NameValue
    ActiveACTIVE
    CreatingCREATING
    CreateFailedCREATE_FAILED
    UpdatingUPDATING
    UpdateFailedUPDATE_FAILED
    DeletingDELETING
    NameValue
    ActiveACTIVE
    CreatingCREATING
    CreateFailedCREATE_FAILED
    UpdatingUPDATING
    UpdateFailedUPDATE_FAILED
    DeletingDELETING
    NameValue
    ACTIVEACTIVE
    CREATINGCREATING
    CREATE_FAILEDCREATE_FAILED
    UPDATINGUPDATING
    UPDATE_FAILEDUPDATE_FAILED
    DELETINGDELETING
    Value
    ACTIVE
    CREATING
    CREATE_FAILED
    UPDATING
    UPDATE_FAILED
    DELETING

    EvaluatorType

    The kind of evaluator resource. Builtin and ThirdParty are AWS-managed global evaluators that you reference but don't create; Custom, CustomCode, and CustomDerived are evaluators that you create.

    NameValue
    BuiltinBuiltin
    ThirdPartyThirdParty
    CustomCustom
    CustomCodeCustomCode
    CustomDerivedCustomDerived
    NameValue
    EvaluatorTypeBuiltinBuiltin
    EvaluatorTypeThirdPartyThirdParty
    EvaluatorTypeCustomCustom
    EvaluatorTypeCustomCodeCustomCode
    EvaluatorTypeCustomDerivedCustomDerived
    Value
    Builtin
    ThirdParty
    Custom
    CustomCode
    CustomDerived
    NameValue
    BuiltinBuiltin
    ThirdPartyThirdParty
    CustomCustom
    CustomCodeCustomCode
    CustomDerivedCustomDerived
    NameValue
    BuiltinBuiltin
    ThirdPartyThirdParty
    CustomCustom
    CustomCodeCustomCode
    CustomDerivedCustomDerived
    NameValue
    BUILTINBuiltin
    THIRD_PARTYThirdParty
    CUSTOMCustom
    CUSTOM_CODECustomCode
    CUSTOM_DERIVEDCustomDerived
    Value
    Builtin
    ThirdParty
    Custom
    CustomCode
    CustomDerived

    Tag, TagArgs

    A set of tags to apply to the resource.

    Key string
    The key name of the tag
    Value string
    The value of the tag
    Key string
    The key name of the tag
    Value string
    The value of the tag
    key string
    The key name of the tag
    value string
    The value of the tag
    key String
    The key name of the tag
    value String
    The value of the tag
    key string
    The key name of the tag
    value string
    The value of the tag
    key str
    The key name of the tag
    value str
    The value of the tag
    key String
    The key name of the tag
    value String
    The value of the tag

    Package Details

    Repository
    AWS Native pulumi/pulumi-aws-native
    License
    Apache-2.0
    aws-native logo aws-native logo AWS Cloud Control

    We recommend new projects start with resources from the AWS provider.

    Viewing docs for AWS Cloud Control v1.86.0
    published on Wednesday, Nov 18, 2026 by Pulumi

      Try Pulumi Cloud free.
      Your team will thank you.

      Start free trial