One option is to consider plugging your speaker or display into a smart plug.
Most smart speakers and smart displays have physical mute buttons, but you might forget to unmute them, which can also be a hassle if you're not nearby. One option is to consider plugging your speaker or display into a smart plug. That way, you can schedule times to turn them off when they're least used.