You can simply insert this in the header of the site using the tools Magento provides.
<meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests">
Where you add this depends on the version of Magento 2 you are using:
Older Versions
Store > Configuration > Web > HTML head > Miscellaneous Scripts >
In newer versions
Content > Design > Configuration > Edit your global view > HTML Head > Scripts and Style Sheets