KB5046740 manages to solve a lot of issues on Windows 11, including some related to Taskbar and File Explorer.
In the Settings under the Personalization > Taskbar menu, you’ve always been able to hide and show various taskbar elements.