Instance types are way too small

[dims]

We need larger instance types for running more pods. Let's not constrict ourselves right now.

Also more pods, needs more ENI(s), especially for AWS VPC CNI, or we run into ContainerCreating issue where pods are stuck. Context:

• https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/troubleshooting.md
• https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/eni-and-ip-target.md
• https://webera.blog/eks-aws-default-vpc-cni-tuning-for-small-networks-a81cbfc48549
• https://github.com/aws/aws-eks-best-practices/blob/master/content/networking/vpc-cni/index.md
• https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html

here's what we see in build logs:

 Aug 16 11:53:17.443: INFO: At 2023-08-16 11:51:56 +0000 UTC - event for httpd: {kubelet ip-172-31-34-156} FailedCreatePodSandBox: Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2c584e052a55d7c162d155a1d607c5cccf658ae6232a99c2250da7d8951715d5": plugin type="aws-cni" name="aws-cni" failed (add): add cmd: failed to assign an IP address to container

here's what the ipamd logs show:

{"level":"debug","ts":"2023-08-16T11:51:44.529Z","caller":"datastore/data_store.go:652","msg":"AssignPodIPv4Address: ENI eni-01550f3b4baff6dba does not have available addresses"}
{"level":"error","ts":"2023-08-16T11:51:44.529Z","caller":"datastore/data_store.go:652","msg":"DataStore has no available IP/Prefix addresses"}
{"level":"info","ts":"2023-08-16T11:51:44.529Z","caller":"rpc/rpc.pb.go:713","msg":"Send AddNetworkReply: IPv4Addr , IPv6Addr: , DeviceNumber: -1, err: assignPodIPv4AddressUnsafe: no available IP/Prefix addresses"}
{"level":"info","ts":"2023-08-16T11:51:44.554Z","caller":"rpc/rpc.pb.go:713","msg":"Received AddNetwork for NS /var/run/netns/cni-f89445a4-57f5-cbc3-9223-422660f720c2, Sandbox e36a596da7a9df6b87babf7c08a37c526860bdd45d786a6b82ad9659c1dcc3ef, ifname eth0"}

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [dims]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dims]

Also see:

• https://instances.vantage.sh/aws/ec2/r7g.4xlarge
• https://instances.vantage.sh/aws/ec2/r5d.4xlarge