Parameter |
Choices/Defaults |
Comments |
allocated_capacity
|
|
The number of AWS Glue data processing units (DPUs) to allocate to this Job. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory.
|
aws_access_key
|
|
AWS access key. If not set then the value of the AWS_ACCESS_KEY_ID, AWS_ACCESS_KEY or EC2_ACCESS_KEY environment variable is used.
aliases: ec2_access_key, access_key
|
aws_secret_key
|
|
AWS secret key. If not set then the value of the AWS_SECRET_ACCESS_KEY, AWS_SECRET_KEY, or EC2_SECRET_KEY environment variable is used.
aliases: ec2_secret_key, secret_key
|
command_name
|
Default:
glueetl
|
The name of the job command. This must be 'glueetl'.
|
command_script_location
required |
|
The S3 path to a script that executes a job.
|
connections
|
|
A list of Glue connections used for this job.
|
default_arguments
|
|
A dict of default arguments for this job. You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.
|
description
|
|
Description of the job being defined.
|
ec2_url
|
|
Url to use to connect to EC2 or your Eucalyptus cloud (by default the module will use EC2 endpoints). Ignored for modules where region is required. Must be specified for all other modules if region is not used. If not set then the value of the EC2_URL environment variable, if any, is used.
|
max_concurrent_runs
|
|
The maximum number of concurrent runs allowed for the job. The default is 1. An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.
|
max_retries
|
|
The maximum number of times to retry this job if it fails.
|
name
required |
|
The name you assign to this job definition. It must be unique in your account.
|
profile
(added in 1.6) |
|
Uses a boto profile. Only works with boto >= 2.24.0.
|
region
|
|
aliases: aws_region, ec2_region
|
role
required |
|
The name or ARN of the IAM role associated with this job.
|
security_token
(added in 1.6) |
|
AWS STS security token. If not set then the value of the AWS_SECURITY_TOKEN or EC2_SECURITY_TOKEN environment variable is used.
aliases: access_token
|
state
required |
|
Create or delete the AWS Glue job.
|
timeout
|
|
The job timeout in minutes.
|
validate_certs
bool
(added in 1.5) |
|
When set to "no", SSL certificates will not be validated for boto versions >= 2.6.0.
|