Satellite IDs Break Audio

DustyArmstrong · June 18, 2021, 8:25pm

Hi there,

I had some issues earlier with returning speech, with one aspect to fix being the Satellite Site IDs. It looks like changing the satellite ID to anything but default stops audio from being played on the satellite. My satellite is set to handle Wake, Audio Recording, Audio Playing etc.

I receive a complete timeout normally, but if (on the base server) I select Remote HTTP for audio and input the IP for my satellite http://1.2.3.4:12101/api/play-wav I receive the below. Leaving both the base server and satellite as default resolves the issue, though I still can’t play audio from the base to the satellite.

[ERROR:2021-06-18 20:19:54,593] rhasspyserver_hermes: 'utf-8' codec can't decode byte 0xa4 in position 4: invalid start byte
Traceback (most recent call last):
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1821, in full_dispatch_request
    result = await self.dispatch_request(request_context)
  File "/usr/lib/rhasspy/.venv/lib/python3.7/site-packages/quart/app.py", line 1869, in dispatch_request
    return await handler(**request_.view_args)
  File "/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/__main__.py", line 1114, in api_play_wav
    url_or_path = data.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa4 in position 4: invalid start byte

Does anyone know why this happens? Is it because I am using HTTP instead of MQTT?

EDIT: Just to ask as well - what is the best way to test the functionality?

Also, my logs from Home Assistant for the intent handle and returned speech:

[DEBUG:2021-06-18 21:38:23,279] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=f25ef33f-eda8-4f05-ba45-a0cad032ee08)
[DEBUG:2021-06-18 21:38:20,969] rhasspyserver_hermes: Publishing 160 bytes(s) to hermes/tts/say
[DEBUG:2021-06-18 21:38:20,969] rhasspyserver_hermes: -> TtsSay(text='test audio', site_id='livingroom', lang=None, id='992e3fdb-06c3-4a18-a115-0bacb02e5c05', session_id='', volume=1.0)
[DEBUG:2021-06-18 21:38:20,968] rhasspyserver_hermes: TTS timeout will be 30 second(s)
[DEBUG:2021-06-18 21:38:20,959] rhasspyserver_hermes: Sent 371 char(s) to websocket
[DEBUG:2021-06-18 21:38:20,957] rhasspyserver_hermes: Handling NluIntent (topic=hermes/intent/GetTest, id=2d43992d-00ad-4efd-88ce-836c7c3828d3)
[DEBUG:2021-06-18 21:38:20,910] rhasspyserver_hermes: Publishing 275 bytes(s) to hermes/nlu/query
[DEBUG:2021-06-18 21:38:20,910] rhasspyserver_hermes: -> NluQuery(input='test input', site_id='livingroom', id='588cd71f-ff8f-4b84-bd2d-9cb39a7107cb', intent_filter=None, session_id='588cd71f-ff8f-4b84-bd2d-9cb39a7107cb', wakeword_id=None, lang=None, custom_data=None, asr_confidence=None, custom_entities=None)
[DEBUG:2021-06-18 21:38:19,606] rhasspyserver_hermes: Handling AsrTextCaptured (topic=hermes/asr/textCaptured, id=e221af15-85b5-450b-bc4a-62bba811b830)
[DEBUG:2021-06-18 21:38:19,358] rhasspyserver_hermes: Publishing 77 bytes(s) to hermes/asr/stopListening
[DEBUG:2021-06-18 21:38:19,358] rhasspyserver_hermes: -> AsrStopListening(site_id='livingroom', session_id='7e74573f-6e4a-4538-a0c6-0c063da1c354')
[DEBUG:2021-06-18 21:38:19,358] rhasspyserver_hermes: Sent 46344 byte(s) of WAV data
[DEBUG:2021-06-18 21:38:19,356] rhasspyserver_hermes: Publishing 184 bytes(s) to hermes/asr/startListening
[DEBUG:2021-06-18 21:38:19,356] rhasspyserver_hermes: -> AsrStartListening(site_id='livingroom', session_id='7e74573f-6e4a-4538-a0c6-0c063da1c354', lang=None, stop_on_silence=False, send_audio_captured=True, wakeword_id=None, intent_filter=None)

romkabouter · June 18, 2021, 9:22pm

This endpoint is expecting one of the options found here:

I don’t know exactly what is send to the endpoint from the base, but non of the above in a correct manner I assume.

DustyArmstrong · June 18, 2021, 9:55pm

Ah, I may have found at least one problem then. I’m trying to do TTS, not a WAV. I changed the Remote HTTP to just http://1.2.3.4:12101, though I still get no sound.

The errors keep changing so I’m not sure. If I try and test it via the web UI it gives me:

[DEBUG:2021-06-18 21:54:05,759] rhasspyserver_hermes: Handling TtsSayFinished (topic=hermes/tts/sayFinished, id=364a6edc-e43e-46ce-bab5-c219d86bd9d1)
[DEBUG:2021-06-18 21:54:05,745] rhasspyserver_hermes: Handling AudioPlayBytes (topic=hermes/audioServer/livingroom/playBytes/5dec737c-a6f3-4959-8392-ced3b2eeeea7, id=364a6edc-e43e-46ce-bab5-c219d86bd9d1)
[DEBUG:2021-06-18 21:54:04,340] rhasspyserver_hermes: Publishing 141 bytes(s) to hermes/tts/say
[DEBUG:2021-06-18 21:54:04,339] rhasspyserver_hermes: -> TtsSay(text='test to speak', site_id='livingroom', lang=None, id='5dec737c-a6f3-4959-8392-ced3b2eeeea7', session_id='', volume=1.0)
[DEBUG:2021-06-18 21:54:04,336] rhasspyserver_hermes: TTS timeout will be 30 second(s)

From the documentation, under the HTTP satellite / base section, it doesn’t seem to make any mention of Satellite IDs. I can only assume it doesn’t work with IDs on HTTP, since when I revert it back to default on both it’s able to speak again, just not from Home Assistant.

I suppose I’ll just have to try and move to MQTT if I want any speech feedback, and send events instead of intents as you suggested for the full functionality. I don’t know if the documentation states it already, but it would be good for anyone new to Rhasspy to see that HTTP and intents aren’t best practice.

romkabouter · June 19, 2021, 5:56am

My main reason for not using HTTP is that there is no way to correctly end a session after that an intenthandler (Home Assistant in my case) is done with it.

But HTTP should be possible.

what is the setting for TextToSpeech and AudioPlaying on the base?
what is the setting for TextToSpeech and AudioPlaying on the satellite?

Because I think there is a configuration problem.

DustyArmstrong · June 19, 2021, 3:31pm

Thanks for the reply, I was just a little frustrated.

On the base:

TTS - NanoTTS (Recommended)
Audio Playing - Disabled (but I’ve also tried Remote HTTP)

On the satellite:

TTS - Remote HTTP (http://1.2.3.4:12101/api/text-to-speech)
Audio Playing - aplay (Recommended)

The audio playing on the base works for the wake word / confirmation.

DustyArmstrong · June 19, 2021, 3:48pm

I set TTS to be performed on the satellite, and everything works. The documentation suggested TTS should be done on the base, but I can’t figure out how to relay that TTS data back to the satellite.

As I only have one satellite at the moment it probably isn’t an issue, but I can see this possibly being a problem with multiple satellites, as Home Assistant would need some way to know which satellite sent the command. Is this possible with intents? I recall something about the ability to include extra info, but that may have only been with events.

Is it also possible to use the variables with intents?

For example:

[SwitchPrinter]
printer_state = (on | off) {state}
turn <printer_state> the printer

Can <printer_state> be sent with the intent? Or is the intent purely [SwitchPrinter] with all the logic handled on Home Assistant?

romkabouter · June 19, 2021, 5:07pm

Adding a querystring should work:
http://<baseIP>:12101/api/text-to-speech?siteid=living_room

I have not yet found a way, but also did not investigate very much, this (seemingly) lack of feature is the reason I use events
The other reason is that you do not have to add a new intent to your configuration.yaml and restart HA before you can use it.
You can simply create a new automation with the required event trigger and reload automations.

romkabouter · June 19, 2021, 5:14pm

Yes, the slots are available, you can use in a template like {{ printer_state }}

DustyArmstrong · June 19, 2021, 5:27pm

Thanks for the info, that’s helpful to know.

So with events, it’s possible to determine which satellite the command came from and return the speech accordingly? I’ll need to try then, it’s one of those things though, I’ve built up a fair amount now with intents so to change to events will be a bit painful!

Regarding the slots, how do these work? I know there’s the slots section in Rhasspy, do I need to fill something in there? I assume though that in Home Assistant it would still need a separate entries for printer on and printer off, it’s just Rhasspy that would determine which intent to send? Sorry to ask so many questions.

romkabouter · June 19, 2021, 5:32pm

Yup.

This is my automation for lights, but I use MQTT:
Your slots will be available via trigger.event.data.<slotname>
Also, you have the sessionID as to close the session on Rhasspy

- id: '1581372525473'
  alias: EventLampen
  trigger:
  - event_data: {}
    event_type: rhasspy_Lights
    platform: event
  condition: []
  action:
  - data_template:
      entity_id: light.{{ trigger.event.data.location }}
    service_template: light.turn_{{ trigger.event.data.action }}
  - service: mqtt.publish
    data:
      topic: hermes/dialogueManager/endSession
      payload_template: '{"sessionId": "{{trigger.event.data._intent.sessionId}}",
        "text": "Ok, {{ trigger.event.data.location }} {{trigger.event.data.action}}"}'
  mode: single

I think it should also be available via intents, maybe I have to investigate a bit more

romkabouter · June 19, 2021, 5:34pm

This is a good read for sentences and slots:
https://rhasspy.readthedocs.io/en/latest/training/#sentencesini

If you have a slot named action, you can use a service template to turn the printer on or off, see my example above. But my knowlegde is for now purely based on events and automations.
However, templates can be use in intent_scripts as well

DustyArmstrong · June 19, 2021, 5:44pm

Cool, thanks. I’ll have to start that setup then when I get the willpower. I have the MQTT broker in HA, does that link into the MQTT on Rhasspy at all? Like it all uses the same MQTT “network”.

I’ll have a read and hopefully I’ll figure it out! I would be interested to know more about intents too, like what is literally sent to HA. My logs in HA (for a failed intent) show:

homeassistant.helpers.intent.UnknownIntent: Unknown intent GetTime

That suggested to me that only GetTime is sent from Rhasspy, without anything else, like GetTime {variable}, or something. I definitely use templates in HA for TTS back to Rhasspy, so I know that works, it’s just how to send the variable to be templated you know.

    TellDate:
        action:
        - data_template:
            payload: 'The date is {{ now().day }} {{ (now().strftime("%b")) }} {{ now().year }}'
          service: rest_command.rhasspy_speak

In my head I would imagine it to be something like this in HA (?), where {{printer_state}} is extra command data from Rhasspy:

    SwitchPrinter:
        action:
            - service: switch.turn_{{printer_state}}
              entity_id: switch.printer

No idea though! It does look like yours, but I don’t know if it works with intents. The docs certainly don’t suggest it works, the extra data is only mentioned in the events section.

romkabouter · June 19, 2021, 5:56pm

Yes, set MQTT in Rhasspy to external with the ip adress of your HA instance.

When you change it to service_template it will work

romkabouter · June 19, 2021, 5:58pm

You have TellDate in your code, not GetTIme. Or do you also have GetTime as intentscript?

DustyArmstrong · June 19, 2021, 6:05pm

OK great.

So it should be:

    SwitchPrinter:
        action:
            - service_template: switch.turn_{{printer_state}}
              entity_id: switch.printer

And that should pick up from Rhasspy if I say on or off?

Also the TellDate / GetTime was just an example to give so I could force a log to be generated.

romkabouter · June 19, 2021, 6:06pm

Yes, that should work

DustyArmstrong · June 19, 2021, 6:11pm

So I tried it, and it gave me this error:

Template variable warning: 'printer_state' is undefined when rendering 'turning {{printer_state}} the printer'
Template variable warning: 'printer_state' is undefined when rendering 'switch.turn_{{printer_state}}'

I assume from that then that intents can’t handle variables, only events.

romkabouter · June 19, 2021, 6:14pm

I have checked you sentences, but you should use state, not printer_state.
{state} is the name of your slot. I am trying to find the correct syntax for intents

DustyArmstrong · June 19, 2021, 6:21pm

Ahh, I see now yeah.

I tried it, it went through without error this time! The printer didn’t turn on but that’s probably another issue for HA

romkabouter · June 19, 2021, 6:47pm

I actually found it

I have made a test:

intent_script:
  Locations:
    action:
      - service: mqtt.publish
        data:
          payload_template: '{{ _intent }}'
          topic: homeassistant/test

When I do a Rhasspy look for this intent I get this message on homeassistant/test

So, I think {{ _intent.siteId }} will give you the siteId