How do I set up an edge node for my Amazon EMR cluster?
Last updated: 2019-04-18
How do I create an edge node (also known as a gateway node) for my Amazon EMR cluster?
Short Description
To create an edge node:
- Create an Amazon Machine Image (AMI) for the master node's Amazon Elastic Compute Cloud (Amazon EC2) instance.
- Launch a new EC2 instance from the AMI. Be sure that the new instance's security group allows access to the Amazon EMR cluster's nodes and allows inbound SSH traffic.
- Connect to the EC2 instance using SSH and then submit jobs to the Amazon EMR cluster.
Resolution
- Open the Amazon EMR console.
- Choose the name of the cluster that you want to create an edge node for.
- Choose the Hardware tab, and then choose the ID of the master node. Note the EC2 instance ID—you'll need it in step 5.
- Open the Amazon Elastic Compute Cloud (Amazon EC2) console.
- Choose the master node's instance ID.
- Choose Actions, Image, and then Create Image.
- Enter an Image name and Image description.
- Choose the No reboot option, and then choose Create Image.
- From the navigation pane, choose AMIs.
- When the image's status changes to available, select the image, and then choose Launch.
- Choose an instance type, and then choose Next: Configure Instance Details.
- In Step 3: Configure Instance Details, for IAM role, either choose EMR_EC2_DefaultRole, or choose another role that allows Amazon EMR to call other AWS services.
- Choose Next: Add Storage, Next: Add Tags, Next: Configure Security Group. You can use the default configurations or change them if needed.
- In Step 6: Configure Security Group, choose Create a new security group or Select an existing security group:
For Create a new security group: Be sure that the security group allows access to the Amazon EMR cluster's nodes and allows inbound SSH traffic.
For Select an existing security group: Select the master node's security group (the default name is "ElasticMapReduce-master"), and then select a second security group that allows inbound SSH traffic. - Launch the instance.
- Connect to the EC2 instance using SSH and then submit jobs to the Amazon EMR cluster.
Related Information
Did this article help you?
Anything we could improve?
Need more help?