Nginx should be working fine for http and https now.
The error returning was “The plain HTTP request was sent to HTTPS port” which was caused from having the 80 and 443 ports in the same server block, under the Nginx config. Had to assign the standard HTTP and HTTPS (SSL) ports to a seperate server block which fixes that specific issue.
Everything should be working fine with the website now and no more issues.
Forgot to mention, switched all the downloads to https://dl.dubbeh.net. So everything will be archived in one location.