google-native.datalabeling/v1beta1.EvaluationJob

Name of the AnnotationSpecSet describing all the labels that your machine learning model outputs. You must create this resource before you create an evaluation job and provide its name in the following format: "projects/{project_id}/annotationSpecSets/{annotation_spec_set_id}"

Description of the job. The description can be up to 25,000 characters long.

Configuration details for the evaluation job.

Whether you want Data Labeling Service to provide ground truth labels for prediction input. If you want the service to assign human labelers to annotate your data, set this to true. If you want to provide your own ground truth labels in the evaluation job's BigQuery table, set this to false.

The AI Platform Prediction model version to be evaluated. Prediction input and output is sampled from this model version. When creating an evaluation job, specify the model version in the following format: "projects/{project_id}/models/{model_name}/versions/{version_name}" There can only be one evaluation job per model version.

Describes the interval at which the job runs. This interval must be at least 1 day, and it is rounded to the nearest day. For example, if you specify a 50-hour interval, the job runs every 2 days. You can provide the schedule in crontab format or in an English-like format. Regardless of what you specify, the job will run at 10:00 AM UTC. Only the interval from this schedule is used, not the specific time of day.

Name of the AnnotationSpecSet describing all the labels that your machine learning model outputs. You must create this resource before you create an evaluation job and provide its name in the following format: "projects/{project_id}/annotationSpecSets/{annotation_spec_set_id}"

Description of the job. The description can be up to 25,000 characters long.

Configuration details for the evaluation job.

Whether you want Data Labeling Service to provide ground truth labels for prediction input. If you want the service to assign human labelers to annotate your data, set this to true. If you want to provide your own ground truth labels in the evaluation job's BigQuery table, set this to false.

The AI Platform Prediction model version to be evaluated. Prediction input and output is sampled from this model version. When creating an evaluation job, specify the model version in the following format: "projects/{project_id}/models/{model_name}/versions/{version_name}" There can only be one evaluation job per model version.

Describes the interval at which the job runs. This interval must be at least 1 day, and it is rounded to the nearest day. For example, if you specify a 50-hour interval, the job runs every 2 days. You can provide the schedule in crontab format or in an English-like format. Regardless of what you specify, the job will run at 10:00 AM UTC. Only the interval from this schedule is used, not the specific time of day.

Name of the AnnotationSpecSet describing all the labels that your machine learning model outputs. You must create this resource before you create an evaluation job and provide its name in the following format: "projects/{project_id}/annotationSpecSets/{annotation_spec_set_id}"

Description of the job. The description can be up to 25,000 characters long.

Configuration details for the evaluation job.

Whether you want Data Labeling Service to provide ground truth labels for prediction input. If you want the service to assign human labelers to annotate your data, set this to true. If you want to provide your own ground truth labels in the evaluation job's BigQuery table, set this to false.

The AI Platform Prediction model version to be evaluated. Prediction input and output is sampled from this model version. When creating an evaluation job, specify the model version in the following format: "projects/{project_id}/models/{model_name}/versions/{version_name}" There can only be one evaluation job per model version.

Describes the interval at which the job runs. This interval must be at least 1 day, and it is rounded to the nearest day. For example, if you specify a 50-hour interval, the job runs every 2 days. You can provide the schedule in crontab format or in an English-like format. Regardless of what you specify, the job will run at 10:00 AM UTC. Only the interval from this schedule is used, not the specific time of day.

Name of the AnnotationSpecSet describing all the labels that your machine learning model outputs. You must create this resource before you create an evaluation job and provide its name in the following format: "projects/{project_id}/annotationSpecSets/{annotation_spec_set_id}"

Description of the job. The description can be up to 25,000 characters long.

Configuration details for the evaluation job.

Whether you want Data Labeling Service to provide ground truth labels for prediction input. If you want the service to assign human labelers to annotate your data, set this to true. If you want to provide your own ground truth labels in the evaluation job's BigQuery table, set this to false.

The AI Platform Prediction model version to be evaluated. Prediction input and output is sampled from this model version. When creating an evaluation job, specify the model version in the following format: "projects/{project_id}/models/{model_name}/versions/{version_name}" There can only be one evaluation job per model version.

Describes the interval at which the job runs. This interval must be at least 1 day, and it is rounded to the nearest day. For example, if you specify a 50-hour interval, the job runs every 2 days. You can provide the schedule in crontab format or in an English-like format. Regardless of what you specify, the job will run at 10:00 AM UTC. Only the interval from this schedule is used, not the specific time of day.

Name of the AnnotationSpecSet describing all the labels that your machine learning model outputs. You must create this resource before you create an evaluation job and provide its name in the following format: "projects/{project_id}/annotationSpecSets/{annotation_spec_set_id}"

Description of the job. The description can be up to 25,000 characters long.

Configuration details for the evaluation job.

Whether you want Data Labeling Service to provide ground truth labels for prediction input. If you want the service to assign human labelers to annotate your data, set this to true. If you want to provide your own ground truth labels in the evaluation job's BigQuery table, set this to false.

The AI Platform Prediction model version to be evaluated. Prediction input and output is sampled from this model version. When creating an evaluation job, specify the model version in the following format: "projects/{project_id}/models/{model_name}/versions/{version_name}" There can only be one evaluation job per model version.

Describes the interval at which the job runs. This interval must be at least 1 day, and it is rounded to the nearest day. For example, if you specify a 50-hour interval, the job runs every 2 days. You can provide the schedule in crontab format or in an English-like format. Regardless of what you specify, the job will run at 10:00 AM UTC. Only the interval from this schedule is used, not the specific time of day.

Name of the AnnotationSpecSet describing all the labels that your machine learning model outputs. You must create this resource before you create an evaluation job and provide its name in the following format: "projects/{project_id}/annotationSpecSets/{annotation_spec_set_id}"

Description of the job. The description can be up to 25,000 characters long.

Configuration details for the evaluation job.

Whether you want Data Labeling Service to provide ground truth labels for prediction input. If you want the service to assign human labelers to annotate your data, set this to true. If you want to provide your own ground truth labels in the evaluation job's BigQuery table, set this to false.

The AI Platform Prediction model version to be evaluated. Prediction input and output is sampled from this model version. When creating an evaluation job, specify the model version in the following format: "projects/{project_id}/models/{model_name}/versions/{version_name}" There can only be one evaluation job per model version.

Describes the interval at which the job runs. This interval must be at least 1 day, and it is rounded to the nearest day. For example, if you specify a 50-hour interval, the job runs every 2 days. You can provide the schedule in crontab format or in an English-like format. Regardless of what you specify, the job will run at 10:00 AM UTC. Only the interval from this schedule is used, not the specific time of day.

Every time the evaluation job runs and an error occurs, the failed attempt is appended to this array.

Timestamp of when this evaluation job was created.

The provider-assigned unique ID for this managed resource.

After you create a job, Data Labeling Service assigns a name to the job with the following format: "projects/{project_id}/evaluationJobs/ {evaluation_job_id}"

Describes the current state of the job.

Every time the evaluation job runs and an error occurs, the failed attempt is appended to this array.

Timestamp of when this evaluation job was created.

The provider-assigned unique ID for this managed resource.

After you create a job, Data Labeling Service assigns a name to the job with the following format: "projects/{project_id}/evaluationJobs/ {evaluation_job_id}"

Describes the current state of the job.

Every time the evaluation job runs and an error occurs, the failed attempt is appended to this array.

Timestamp of when this evaluation job was created.

The provider-assigned unique ID for this managed resource.

After you create a job, Data Labeling Service assigns a name to the job with the following format: "projects/{project_id}/evaluationJobs/ {evaluation_job_id}"

Describes the current state of the job.

Every time the evaluation job runs and an error occurs, the failed attempt is appended to this array.

Timestamp of when this evaluation job was created.

The provider-assigned unique ID for this managed resource.

After you create a job, Data Labeling Service assigns a name to the job with the following format: "projects/{project_id}/evaluationJobs/ {evaluation_job_id}"

Describes the current state of the job.

Every time the evaluation job runs and an error occurs, the failed attempt is appended to this array.

Timestamp of when this evaluation job was created.

The provider-assigned unique ID for this managed resource.

After you create a job, Data Labeling Service assigns a name to the job with the following format: "projects/{project_id}/evaluationJobs/ {evaluation_job_id}"

Describes the current state of the job.

Every time the evaluation job runs and an error occurs, the failed attempt is appended to this array.

Timestamp of when this evaluation job was created.

The provider-assigned unique ID for this managed resource.

After you create a job, Data Labeling Service assigns a name to the job with the following format: "projects/{project_id}/evaluationJobs/ {evaluation_job_id}"

Describes the current state of the job.

Details of errors that occurred.

Details of errors that occurred.

Details of errors that occurred.

Details of errors that occurred.

Details of errors that occurred.

Details of errors that occurred.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

BigQuery URI to a table, up to 2,000 characters long. If you specify the URI of a table that does not exist, Data Labeling Service creates a table at the URI with the correct schema when you create your EvaluationJob. If you specify the URI of a table that already exists, it must have the correct schema. Provide the table URI in the following format: "bq://{your_project_id}/ {your_dataset_name}/{your_table_name}" Learn more.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Minimum intersection-over-union (IOU) required for 2 bounding boxes to be considered a match. This must be a number between 0 and 1.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Annotation spec set resource name.

Optional. Instruction message showed on contributors UI.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Whether the classification task is multi-label or not.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

Only specify this field if the related model performs image object detection (IMAGE_BOUNDING_BOX_ANNOTATION). Describes how to evaluate bounding boxes.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

An email address to send alerts to.

A number between 0 and 1 that describes a minimum mean average precision threshold. When the evaluation job runs, if it calculates that your model version's predictions from the recent interval have meanAveragePrecision below this threshold, then it sends an alert to your specified email.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field: * data_json_key: the data key for prediction input. You must provide either this key or reference_json_key. * reference_json_key: the data reference key for prediction input. You must provide either this key or data_json_key. * label_json_key: the label key for prediction output. Required. * label_score_json_key: the score key for prediction output. Required. * bounding_box_json_key: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys.

Specify this field if your model version performs image object detection (bounding box detection). annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet.

Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify the boundingBoxEvaluationOptions field within this configuration. Otherwise, provide an empty object for this configuration.

Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run.

The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overrides example_sample_percentage: even if the service has not sampled enough predictions to fulfill example_sample_perecentage during an interval, it stops sampling predictions when it meets this limit.

Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery.

Optional. Details for human annotation of your data. If you set labelMissingGroundTruth to true for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in the instruction field within this configuration.

Specify this field if your model version performs image classification or general classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields: * dataType must be one of IMAGE, TEXT, or GENERAL_DATA. * annotationType must be one of IMAGE_CLASSIFICATION_ANNOTATION, TEXT_CLASSIFICATION_ANNOTATION, GENERAL_CLASSIFICATION_ANNOTATION, or IMAGE_BOUNDING_BOX_ANNOTATION (image object detection). * If your machine learning model performs classification, you must specify classificationMetadata.isMultiLabel. * You must specify bigquerySource (not gcsSource).

Specify this field if your model version performs text classification. annotationSpecSet in this configuration must match EvaluationJob.annotationSpecSet. allowMultiLabel in this configuration must match classificationMetadata.isMultiLabel in input_config.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

The input URI of source file. This must be a Cloud Storage path (gs://...).

The format of the source file. Only "text/csv" is supported.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Instruction resource name.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Instruction resource name.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Instruction resource name.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Instruction resource name.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Instruction resource name.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Instruction resource name.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Instruction resource name.

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Instruction resource name.

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Instruction resource name.

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Instruction resource name.

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Instruction resource name.

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

Optional. A human-readable description for AnnotatedDataset. The description can be up to 10000 characters long.

A human-readable name for AnnotatedDataset defined by users. Maximum of 64 characters .

Optional. If you want your own labeling contributors to manage and work on this labeling request, you can set these contributors here. We will give them access to the question types in crowdcompute. Note that these emails must be registered in crowdcompute worker UI: https://crowd-compute.appspot.com/

Instruction resource name.

Optional. A human-readable label used to logically group labeling tasks. This string must match the regular expression [a-zA-Z\\d_-]{0,128}.

Optional. The Language of this question, as a BCP-47. Default value is en-US. Only need to set this when task is language related. For example, French text classification.

Optional. Maximum duration for contributors to answer a question. Maximum is 3600 seconds. Default is 3600 seconds.

Optional. Replication of questions. Each question will be sent to up to this number of contributors to label. Aggregated answers will be returned. Default is set to 1. For image related labeling, valid values are 1, 3, 5.

Email of the user who started the labeling task and should be notified by email. If empty no notification will be sent.

Annotation spec set resource name.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Optional. The type of how to aggregate answers.

Annotation spec set resource name.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Optional. The type of how to aggregate answers.

Annotation spec set resource name.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Optional. The type of how to aggregate answers.

Annotation spec set resource name.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Optional. The type of how to aggregate answers.

Annotation spec set resource name.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Optional. The type of how to aggregate answers.

Annotation spec set resource name.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Optional. The type of how to aggregate answers.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Annotation spec set resource name.

Optional. The type of how to aggregate answers.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Annotation spec set resource name.

Optional. The type of how to aggregate answers.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Annotation spec set resource name.

Optional. The type of how to aggregate answers.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Annotation spec set resource name.

Optional. The type of how to aggregate answers.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Annotation spec set resource name.

Optional. The type of how to aggregate answers.

Optional. If allow_multi_label is true, contributors are able to choose multiple labels for one image.

Annotation spec set resource name.

Optional. The type of how to aggregate answers.

Data type must be specifed when user tries to import data.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Data type must be specifed when user tries to import data.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Data type must be specifed when user tries to import data.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Data type must be specifed when user tries to import data.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Data type must be specifed when user tries to import data.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Data type must be specifed when user tries to import data.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Data type must be specifed when user tries to import data.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Data type must be specifed when user tries to import data.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Data type must be specifed when user tries to import data.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Data type must be specifed when user tries to import data.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Data type must be specifed when user tries to import data.

Source located in Cloud Storage.

Required for text import, as language code must be specified.

Optional. The type of annotation to be performed on this data. You must specify this field if you are using this InputConfig in an EvaluationJob.

Source located in BigQuery. You must specify this field if you are using this InputConfig in an EvaluationJob.

Optional. Metadata about annotations for the input. You must specify this field if you are using this InputConfig in an EvaluationJob for a model version that performs classification.

Data type must be specifed when user tries to import data.

Source located in Cloud Storage.