The same behavior can be observed with the Canvas UI Text object in your scene.
For instance, if you set the resolution to 640 X 480 and use full screen mode, even the Unity logo is all blurry. This is simply because everything is rendered at 640 x 480 and ends up being scaled to whatever full screen resolution is which in my case is 3840 x 2160.

Switching to fullscreen doesn't change the target resolution. So unless you change the resolution to match full screen, you'll keep getting this scaling.