With non-TeX fonts, you can select a 'Non-TeX Font Default' math font, which simply loads unicode-math without actually selecting a math font, this then uses the default math otf font, currently Latin Modern. Other fonts still need to be set manually in the preamble, via \setmathfont.
The implementation suppresses unneeded package requests from unicodesymbols, but the output still uses macros instead of full unicode (both is possible with unicode-math).
The whole thing is a proof of concept, and it needs to be tested. I have tested it with the math manual, which compiles and seems to display correctly if I remove some hardcoded package loadings. OTOH I have not much experience with math.
This addresses #7449 partly.
8bf5d4b461