Rolyan, are you using the current version Rhasspy in Base + Satellite configuration ? It sounds as though you are thinking of an old version which only ran stand-alone.
Huh ? If your Base station is a powerful server to give the processing power and audio quality you expect, then surely you don’t want a bulky noisy server sitting next to your living room TV. Home Assistant OS certainly seems to be taking the approach that devices should connect through the LAN, which allows the server to sit in a comms cabinet or back room.
My limited experience of 3 Satellites is that RasPi Zero is too slow to recommend, but RasPi 3A+ and 3B work OK with reSpeaker HATs and Rhasspy (microphone, speaker, LAN and Porcupine wakeword modules only). I doubt that using RasPi 4 for a Satellite would be worth the extra cost. There are other boards with DSP - but at much higher price. ESP32-S3 with DSP sounds much better than any of the current options - I can’t wait !
For Base station, my RasPi 4 (4GB) runs Home Assistant OS with MQTT and Rhasspy add-on (using STT, intent recognition and TTS modules only). I only have a 2-bedroom apartment with few devices connected to HA, and performance is OK. More important is knowing that I can easily upgrade to a NUC or server
From your years of audio industry experience, probably you expect a much higher audio quality than my “OK” Lets face it - no local solution will ever be able to apply enough processing power to compete with the results from Amazon/Google/etc. But that is not the objective of Rhasspy.
I see current Rhasspy as a modular framework with multiple options for each stage of voice processing. And Raspberry Pi is just a convenient low cost way for people to start with Home Assistant and Rhasspy. I expect that if/when ESP32-S3 is interfaced to Rhasspy (using MQTT or another protocol) it will quickly become the most recommended Satellite hardware.
I have absolutely no experience with ESP hardware, so a lot of the second half of your post went over my head ESP32-S3-BOX has a nice little demo, but there’s a huge gap between turning one LED on and off, and running a whole house. Of course it is a development kit, meaning it is up to others to develop the software. Waiting to see how ESP32-S3 progresses…
and so we get back to the same basic issue as DSP on RasPi & reSpeaker. Software is the key.
Sounds great, but (a) is this available now as open source, and (b) does it run locally - or locks us into dependency on Google’s massive cloud processing ?
At the same time, this is an opportunity to re-consider what might be a better control structure suitable for ESP32 satellites, and how can it be implemented.