mardi 20 décembre 2016

Download and Socket timeout on EC2

I recently encountered an interesting problem on EC2:
- 2 machines with the same configuration, running the same software
- one is in a private subnet
- one in a public subnet
- none has a public IP
- private subnet use AWS NAT Gateway  (created in VPC panel) to access internet
- public subnet uses AWS Internet Gateway

Problem: on the machine in the private subnet, the download of a big file stalls after downloading a big part of the file and provokes a socket timeout.

Observations:
- the machine on the public subnet has no problem downloading the big files
- both machine can access internet without any issue

After finding that NAT gateway can slow down connection and given the obvious subnet difference between the 2 machines. I recreated the first machine in a public subnet and... now it's able to download big files without issue.

Hypothesis:
After a while the AWS NAT gateway throttle the bandwidth up to choking the connection which creates a time out.