I struggled with this for a few days before figuring it out, so I’ll post it here in hopes it saves someone a few minutes. When you install puppet and start the puppetmaster (webrick or rack-enabled) it generates a ssl cert for that machine and also generates a CA that you will use to sign all of your clients.
Recent versions of puppet do not add subjectAltNames to the server certificate when it’s generated by the puppetmaster process. This means that if you do not use the same name as your masters hostname to connect to puppet you will get a lovely cert mismatch. I posted a question on serverfault about this (here). It looks like the common practice for EC2 in particular is to use a uuid as the certname for each puppet client. This avoids name collisions and problems with hostnames changing everytime the instance is rebooted. It’s a little harder to keep track of since they aren’t very easy to remember, so caveat emptor.