Putty comes with a tool called PuttyGen, which is used to convert RSA keys to Windows format.
Open PuttyGen, Click on Conversions menu and select Import Key.
Browse to the folder that contains the .pem
When finished, click the 'Save private key' button to save it into a .ppk file.
Open PuTTY and navigate to 'Connection > SSH > Auth'.
Click the Browse button near the 'Private key file for authentication' field.
Select the .ppk file you generated and then click open.
Navigate to the 'Session' category and enter the IP Address of your server in the 'Host Name' field and connect.