Putty comes with a tool called PuttyGen, which is used to convert RSA keys to Windows format.
Open PuttyGen, Click on Conversions menu and select Import Key.
Browse to the folder that contains the .pem
When finished, click the 'Save private key' button to save it into a .ppk file.
If you need to revert the conversion from Linux you can
apt install putty-toolsand then
puttygen server.ppk -O private-openssh -o server.pem
Open PuTTY and navigate to 'Connection > SSH > Auth'.
Click the Browse button near the 'Private key file for authentication' field.
Select the .ppk file you generated and then click open.
Navigate to the 'Session' category and enter the IP Address of your server in the 'Host Name' field and connect.