1. Packages
  2. AWS Cloud Control
  3. API Docs
  4. bedrockagentcore
  5. Evaluator

We recommend new projects start with resources from the AWS provider.

Viewing docs for AWS Cloud Control v1.57.0
published on Monday, Mar 9, 2026 by Pulumi
Schema (JSON)
pulumi/pulumi-aws-native

aws-native.bedrockagentcore.Evaluator

aws-native logo

We recommend new projects start with resources from the AWS provider.

Viewing docs for AWS Cloud Control v1.57.0
published on Monday, Mar 9, 2026 by Pulumi
Schema (JSON)
pulumi/pulumi-aws-native

    Resource Type definition for AWS::BedrockAgentCore::Evaluator - Creates a custom evaluator for agent quality assessment using LLM-as-a-Judge configurations.

    Create Evaluator Resource

    Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

    Constructor syntax

    new Evaluator(name: string, args: EvaluatorArgs, opts?: CustomResourceOptions);
    @overload
def Evaluator(resource_name: str,
              args: EvaluatorArgs,
              opts: Optional[ResourceOptions] = None)

@overload
def Evaluator(resource_name: str,
              opts: Optional[ResourceOptions] = None,
              evaluator_config: Optional[EvaluatorConfigArgs] = None,
              level: Optional[EvaluatorLevel] = None,
              description: Optional[str] = None,
              evaluator_name: Optional[str] = None,
              tags: Optional[Sequence[_root_inputs.TagArgs]] = None)
    func NewEvaluator(ctx *Context, name string, args EvaluatorArgs, opts ...ResourceOption) (*Evaluator, error)
    public Evaluator(string name, EvaluatorArgs args, CustomResourceOptions? opts = null)
    public Evaluator(String name, EvaluatorArgs args)
public Evaluator(String name, EvaluatorArgs args, CustomResourceOptions options)

    type: aws-native:bedrockagentcore:Evaluator
properties: # The arguments to resource properties.
options: # Bag of options to control resource's behavior.

    Parameters

    name string
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    opts CustomResourceOptions
    Bag of options to control resource's behavior.
    resource_name str
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    opts ResourceOptions
    Bag of options to control resource's behavior.
    ctx Context
    Context object for the current deployment.
    name string
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    opts ResourceOption
    Bag of options to control resource's behavior.
    name string
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    opts CustomResourceOptions
    Bag of options to control resource's behavior.
    name String
    The unique name of the resource.
    args EvaluatorArgs
    The arguments to resource properties.
    options CustomResourceOptions
    Bag of options to control resource's behavior.

    Evaluator Resource Properties

    To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

    Inputs

    In Python, inputs that are objects can be passed either as argument classes or as dictionary literals.

    The Evaluator resource accepts the following input properties:

    EvaluatorConfig Pulumi.AwsNative.BedrockAgentCore.Inputs.EvaluatorConfig
    The configuration for the evaluator.
    Level Pulumi.AwsNative.BedrockAgentCore.EvaluatorLevel
    The evaluation level that determines the scope of evaluation.
    Description string
    The description of the evaluator.
    EvaluatorName string
    The name of the evaluator. Must be unique within your account.
    Tags List<Pulumi.AwsNative.Inputs.Tag>
    A list of tags to assign to the evaluator.
    EvaluatorConfig EvaluatorConfigArgs
    The configuration for the evaluator.
    Level EvaluatorLevel
    The evaluation level that determines the scope of evaluation.
    Description string
    The description of the evaluator.
    EvaluatorName string
    The name of the evaluator. Must be unique within your account.
    Tags TagArgs
    A list of tags to assign to the evaluator.
    evaluatorConfig EvaluatorConfig
    The configuration for the evaluator.
    level EvaluatorLevel
    The evaluation level that determines the scope of evaluation.
    description String
    The description of the evaluator.
    evaluatorName String
    The name of the evaluator. Must be unique within your account.
    tags List<Tag>
    A list of tags to assign to the evaluator.
    evaluatorConfig EvaluatorConfig
    The configuration for the evaluator.
    level EvaluatorLevel
    The evaluation level that determines the scope of evaluation.
    description string
    The description of the evaluator.
    evaluatorName string
    The name of the evaluator. Must be unique within your account.
    tags Tag[]
    A list of tags to assign to the evaluator.
    evaluator_config EvaluatorConfigArgs
    The configuration for the evaluator.
    level EvaluatorLevel
    The evaluation level that determines the scope of evaluation.
    description str
    The description of the evaluator.
    evaluator_name str
    The name of the evaluator. Must be unique within your account.
    tags Sequence[TagArgs]
    A list of tags to assign to the evaluator.
    evaluatorConfig Property Map
    The configuration for the evaluator.
    level "TOOL_CALL" | "TRACE" | "SESSION"
    The evaluation level that determines the scope of evaluation.
    description String
    The description of the evaluator.
    evaluatorName String
    The name of the evaluator. Must be unique within your account.
    tags List<Property Map>
    A list of tags to assign to the evaluator.

    Outputs

    All input properties are implicitly available as output properties. Additionally, the Evaluator resource produces the following output properties:

    CreatedAt string
    The timestamp when the evaluator was created.
    EvaluatorArn string
    The Amazon Resource Name (ARN) of the evaluator.
    EvaluatorId string
    The unique identifier of the evaluator.
    Id string
    The provider-assigned unique ID for this managed resource.
    Status Pulumi.AwsNative.BedrockAgentCore.EvaluatorStatus
    The current status of the evaluator.
    UpdatedAt string
    The timestamp when the evaluator was last updated.
    CreatedAt string
    The timestamp when the evaluator was created.
    EvaluatorArn string
    The Amazon Resource Name (ARN) of the evaluator.
    EvaluatorId string
    The unique identifier of the evaluator.
    Id string
    The provider-assigned unique ID for this managed resource.
    Status EvaluatorStatus
    The current status of the evaluator.
    UpdatedAt string
    The timestamp when the evaluator was last updated.
    createdAt String
    The timestamp when the evaluator was created.
    evaluatorArn String
    The Amazon Resource Name (ARN) of the evaluator.
    evaluatorId String
    The unique identifier of the evaluator.
    id String
    The provider-assigned unique ID for this managed resource.
    status EvaluatorStatus
    The current status of the evaluator.
    updatedAt String
    The timestamp when the evaluator was last updated.
    createdAt string
    The timestamp when the evaluator was created.
    evaluatorArn string
    The Amazon Resource Name (ARN) of the evaluator.
    evaluatorId string
    The unique identifier of the evaluator.
    id string
    The provider-assigned unique ID for this managed resource.
    status EvaluatorStatus
    The current status of the evaluator.
    updatedAt string
    The timestamp when the evaluator was last updated.
    created_at str
    The timestamp when the evaluator was created.
    evaluator_arn str
    The Amazon Resource Name (ARN) of the evaluator.
    evaluator_id str
    The unique identifier of the evaluator.
    id str
    The provider-assigned unique ID for this managed resource.
    status EvaluatorStatus
    The current status of the evaluator.
    updated_at str
    The timestamp when the evaluator was last updated.
    createdAt String
    The timestamp when the evaluator was created.
    evaluatorArn String
    The Amazon Resource Name (ARN) of the evaluator.
    evaluatorId String
    The unique identifier of the evaluator.
    id String
    The provider-assigned unique ID for this managed resource.
    status "ACTIVE" | "CREATING" | "CREATE_FAILED" | "UPDATING" | "UPDATE_FAILED" | "DELETING"
    The current status of the evaluator.
    updatedAt String
    The timestamp when the evaluator was last updated.

    Supporting Types

    EvaluatorBedrockEvaluatorModelConfig, EvaluatorBedrockEvaluatorModelConfigArgs

    The configuration for using Amazon Bedrock models in evaluator assessments.
    ModelId string
    The identifier of the Amazon Bedrock model to use for evaluation.
    AdditionalModelRequestFields object
    Additional model-specific request fields.
    InferenceConfig Pulumi.AwsNative.BedrockAgentCore.Inputs.EvaluatorInferenceConfiguration
    ModelId string
    The identifier of the Amazon Bedrock model to use for evaluation.
    AdditionalModelRequestFields interface{}
    Additional model-specific request fields.
    InferenceConfig EvaluatorInferenceConfiguration
    modelId String
    The identifier of the Amazon Bedrock model to use for evaluation.
    additionalModelRequestFields Object
    Additional model-specific request fields.
    inferenceConfig EvaluatorInferenceConfiguration
    modelId string
    The identifier of the Amazon Bedrock model to use for evaluation.
    additionalModelRequestFields any
    Additional model-specific request fields.
    inferenceConfig EvaluatorInferenceConfiguration
    model_id str
    The identifier of the Amazon Bedrock model to use for evaluation.
    additional_model_request_fields Any
    Additional model-specific request fields.
    inference_config EvaluatorInferenceConfiguration
    modelId String
    The identifier of the Amazon Bedrock model to use for evaluation.
    additionalModelRequestFields Any
    Additional model-specific request fields.
    inferenceConfig Property Map

    EvaluatorCategoricalScaleDefinition, EvaluatorCategoricalScaleDefinitionArgs

    A categorical rating scale option.
    Definition string
    The description that explains what this categorical rating represents.
    Label string
    The label of this categorical rating option.
    Definition string
    The description that explains what this categorical rating represents.
    Label string
    The label of this categorical rating option.
    definition String
    The description that explains what this categorical rating represents.
    label String
    The label of this categorical rating option.
    definition string
    The description that explains what this categorical rating represents.
    label string
    The label of this categorical rating option.
    definition str
    The description that explains what this categorical rating represents.
    label str
    The label of this categorical rating option.
    definition String
    The description that explains what this categorical rating represents.
    label String
    The label of this categorical rating option.

    EvaluatorConfig, EvaluatorConfigArgs

    The configuration that defines how an evaluator assesses agent performance.

    EvaluatorInferenceConfiguration, EvaluatorInferenceConfigurationArgs

    The inference configuration parameters that control model behavior during evaluation.
    MaxTokens int
    The maximum number of tokens to generate in the model response.
    Temperature double
    The temperature value that controls randomness in the model's responses.
    TopP double
    The top-p sampling parameter that controls the diversity of the model's responses.
    MaxTokens int
    The maximum number of tokens to generate in the model response.
    Temperature float64
    The temperature value that controls randomness in the model's responses.
    TopP float64
    The top-p sampling parameter that controls the diversity of the model's responses.
    maxTokens Integer
    The maximum number of tokens to generate in the model response.
    temperature Double
    The temperature value that controls randomness in the model's responses.
    topP Double
    The top-p sampling parameter that controls the diversity of the model's responses.
    maxTokens number
    The maximum number of tokens to generate in the model response.
    temperature number
    The temperature value that controls randomness in the model's responses.
    topP number
    The top-p sampling parameter that controls the diversity of the model's responses.
    max_tokens int
    The maximum number of tokens to generate in the model response.
    temperature float
    The temperature value that controls randomness in the model's responses.
    top_p float
    The top-p sampling parameter that controls the diversity of the model's responses.
    maxTokens Number
    The maximum number of tokens to generate in the model response.
    temperature Number
    The temperature value that controls randomness in the model's responses.
    topP Number
    The top-p sampling parameter that controls the diversity of the model's responses.

    EvaluatorLevel, EvaluatorLevelArgs

    ToolCall
    TOOL_CALL
    Trace
    TRACE
    Session
    SESSION
    EvaluatorLevelToolCall
    TOOL_CALL
    EvaluatorLevelTrace
    TRACE
    EvaluatorLevelSession
    SESSION
    ToolCall
    TOOL_CALL
    Trace
    TRACE
    Session
    SESSION
    ToolCall
    TOOL_CALL
    Trace
    TRACE
    Session
    SESSION
    TOOL_CALL
    TOOL_CALL
    TRACE
    TRACE
    SESSION
    SESSION
    "TOOL_CALL"
    TOOL_CALL
    "TRACE"
    TRACE
    "SESSION"
    SESSION

    EvaluatorLlmAsAJudgeEvaluatorConfig, EvaluatorLlmAsAJudgeEvaluatorConfigArgs

    The configuration for LLM-as-a-Judge evaluation.
    Instructions string
    The evaluation instructions that guide the language model in assessing agent performance.
    ModelConfig EvaluatorModelConfig
    RatingScale EvaluatorRatingScale
    instructions String
    The evaluation instructions that guide the language model in assessing agent performance.
    modelConfig EvaluatorModelConfig
    ratingScale EvaluatorRatingScale
    instructions string
    The evaluation instructions that guide the language model in assessing agent performance.
    modelConfig EvaluatorModelConfig
    ratingScale EvaluatorRatingScale
    instructions str
    The evaluation instructions that guide the language model in assessing agent performance.
    model_config EvaluatorModelConfig
    rating_scale EvaluatorRatingScale
    instructions String
    The evaluation instructions that guide the language model in assessing agent performance.
    modelConfig Property Map
    ratingScale Property Map

    EvaluatorModelConfig, EvaluatorModelConfigArgs

    The model configuration that specifies which foundation model to use for evaluation.

    EvaluatorNumericalScaleDefinition, EvaluatorNumericalScaleDefinitionArgs

    A numerical rating scale option.
    Definition string
    The description that explains what this numerical rating represents.
    Label string
    The label that describes this numerical rating option.
    Value double
    The numerical value for this rating scale option.
    Definition string
    The description that explains what this numerical rating represents.
    Label string
    The label that describes this numerical rating option.
    Value float64
    The numerical value for this rating scale option.
    definition String
    The description that explains what this numerical rating represents.
    label String
    The label that describes this numerical rating option.
    value Double
    The numerical value for this rating scale option.
    definition string
    The description that explains what this numerical rating represents.
    label string
    The label that describes this numerical rating option.
    value number
    The numerical value for this rating scale option.
    definition str
    The description that explains what this numerical rating represents.
    label str
    The label that describes this numerical rating option.
    value float
    The numerical value for this rating scale option.
    definition String
    The description that explains what this numerical rating represents.
    label String
    The label that describes this numerical rating option.
    value Number
    The numerical value for this rating scale option.

    EvaluatorRatingScale, EvaluatorRatingScaleArgs

    The rating scale that defines how evaluators should score agent performance.

    EvaluatorStatus, EvaluatorStatusArgs

    Active
    ACTIVE
    Creating
    CREATING
    CreateFailed
    CREATE_FAILED
    Updating
    UPDATING
    UpdateFailed
    UPDATE_FAILED
    Deleting
    DELETING
    EvaluatorStatusActive
    ACTIVE
    EvaluatorStatusCreating
    CREATING
    EvaluatorStatusCreateFailed
    CREATE_FAILED
    EvaluatorStatusUpdating
    UPDATING
    EvaluatorStatusUpdateFailed
    UPDATE_FAILED
    EvaluatorStatusDeleting
    DELETING
    Active
    ACTIVE
    Creating
    CREATING
    CreateFailed
    CREATE_FAILED
    Updating
    UPDATING
    UpdateFailed
    UPDATE_FAILED
    Deleting
    DELETING
    Active
    ACTIVE
    Creating
    CREATING
    CreateFailed
    CREATE_FAILED
    Updating
    UPDATING
    UpdateFailed
    UPDATE_FAILED
    Deleting
    DELETING
    ACTIVE
    ACTIVE
    CREATING
    CREATING
    CREATE_FAILED
    CREATE_FAILED
    UPDATING
    UPDATING
    UPDATE_FAILED
    UPDATE_FAILED
    DELETING
    DELETING
    "ACTIVE"
    ACTIVE
    "CREATING"
    CREATING
    "CREATE_FAILED"
    CREATE_FAILED
    "UPDATING"
    UPDATING
    "UPDATE_FAILED"
    UPDATE_FAILED
    "DELETING"
    DELETING

    Tag, TagArgs

    A set of tags to apply to the resource.
    Key string
    The key name of the tag
    Value string
    The value of the tag
    Key string
    The key name of the tag
    Value string
    The value of the tag
    key String
    The key name of the tag
    value String
    The value of the tag
    key string
    The key name of the tag
    value string
    The value of the tag
    key str
    The key name of the tag
    value str
    The value of the tag
    key String
    The key name of the tag
    value String
    The value of the tag

    Package Details

    Repository
    AWS Native pulumi/pulumi-aws-native
    License
    Apache-2.0
    aws-native logo

    We recommend new projects start with resources from the AWS provider.

    Viewing docs for AWS Cloud Control v1.57.0
    published on Monday, Mar 9, 2026 by Pulumi
    Schema (JSON)
    pulumi/pulumi-aws-native
      Try Pulumi Cloud free. Your team will thank you.