So after hours of initial troubleshooting and some back and forth with Microsoft support, Microsoft support have stated that the Dynamics 365 App for Outlook (NOT the Dynamics 365 outlook client) is NOT supported on outlook 2013 (32bit or 64bit).
I showed them this image initially, found from their site:
But after confirmation with the team, they stated that the link is incorrect and would be updated. They showed me this image stating that only outlook 2016 is supported:
Hope this saves you the hours I spent trying to troubleshoot the app to work on outlook 2013!