Intent handling with command

Development
1

I do not know what I’m doing wrong, thank you to help me, please.

What I did was the following:
Place the example code on the desktop: https://github.com/synesthesiam/rhasspy/blob/master/bin/mock-commands/handle.py

Then in Profile.JSON Place it as and as the documentation says:

"handle": {
  "system": "command",
  "command": {
      "program": "/path/to/program",
      "arguments": []
  }
}

Note: I did it from the web configurations and update profile.json and wrote the same as I had.

To the Handle.py code add this (to realize whether it works or not when playing that sound):

elif intent == "Hello":
    replies = ['Hi!', 'Hello!', 'Hey there!', 'Greetings.']
    os.system('mpg321 /home/pi/Desktop/test.mp3')

I pronounce “Hello”, I detect it on the web but I do not work the code, I do not know if something else has to be done.

2

I continued to test another code. With the other if it ran well, but, I have to have it running it to work.

I guess the ideal thing is not having it always running to work… or I’m wrong?

The code that I speak is as follows:

import websocket
import json
from datetime import datetime
import os
import time

# Intents are passed through here
def on_message(ws, message):
    data = json.loads(message)
    print("**Captured New Intent**")
    print(data)

    if ("Hello" == data["intent"]["name"]):
        os.system(‘mpg321 /home/pi/Desktop/test.mp3’)
        
   
def on_error(ws, error):
    print(error)

def on_close(ws):
    print("\n**Disconnected**\n")

def on_open(ws):
    print("\n**Connected**\n")

# Start web socket client
if __name__ == "__main__":
    ws = websocket.WebSocketApp("ws://localhost:12101/api/events/intent",
                              on_message = on_message,
                              on_error = on_error,
                              on_close = on_close)
    ws.on_open = on_open
    ws.run_forever()
3

Did you change the /path/to/program to actually point to your program?

4

That’s right, I change the path (via profile.json) of the Python file (it’s on the desktop). I said “Hello” and nothing happened, I’m doing something wrong?

Note 1: I change the patch manually from the file (profile.json), and also change it from the web (from the web I only change the patch and the file profile.json was automatically changed).

Note 2: With the 2nd code all the commands are executed correctly via websocket, but I guess it is not the right way to do it, right?

5

If you’re using Docker, make sure that you give the running container access to /path/to/program with a -v /path/to:/path/to in the docker run command.

1 Like
6

Honestly I do not understand what I should do. Imagine that my path is at: /home/pi/desktop/robot.py

What should I execute? (Sorry, I’m new using Docker)

7

Let’s start from the beginning :slight_smile:

What command did you start the Rhasspy docker with?

1 Like
8

Just this: New profile with Docker

9

Ok, with that command this folder:
$HOME/.config/rhasspy/profiles is mapped to /profiles in the container.

If you put robot.py into $HOME/.config/rhasspy/profiles, then you can set this in your profile:
“program”: “/profiles/robot.py”

10

At http://localhost:12101/settings
The option “Intent Handling” Select “Local Command”, in “Program” I wrote: /profiles/robot.py

I keep it and I went straight to the code to see that everything is fine, the following comes out:

"handle": {
    "command": {
        "program": "/profiles/robot.py"
    },
    "system": "command"
},

It does not do anything when I say a phrase, I am waiting for when I call the robot react according to the code (honestly I do not know what the typical operation is).

The code that I put on robot.py was this:

Under “Hello” I add that it reproduces an audio when you recognize it, but nothing happens.

What am I doing wrong? :confused:

11

What is the Rhasspy log?
What do you have as sentences in what language?

1 Like
12

rhasspyserver_hermes: Sent 421 char(s) to websocket
rhasspyserver_hermes: <- NluIntent(input=‘Hello’, intent=Intent(intent_name=‘Hello’, confidence_score=1.0), site_id=‘default’, id=None, slots=[], session_id=‘default-hey_robot’, custom_data=‘hey_robot’, asr_tokens=[[AsrToken(value=‘hello’, confidence=1.0, range_start=0, range_end=5, time=None)]], asr_confidence=0.00010727685683729188, raw_input=‘hello’, wakeword_id=‘hey_robot’, lang=None)

13

This is what comes out in the log, I assure you that the attempt that registers the log is the same as I place in the code:

Note: When I try the second code and I run the Python with Socket works well, the first code I do not run, I guess the first code I should not execute it.

Remember…
This is the 1st code:

This is the 2nd code:

14

With “Execute” I mean that in console I run Python3 + Path. The code 1 I do not run it, I suppose that Rhasspy does it and that’s why I must place the path in profile.json

The code 2 (the one you have socket) if I run and when I pronounced a phrase the robot recognizes it and runs the conditional that places in the code.

I say it because I want it to be clear what I do, so you can help me with a better precision. Also, because I do not know what “intent handling” of Rhaspy, so he would appreciate it if you explain me, please.

15

I could not find it in your text, but did you put robot.py here?

16

Yes, the file “robot.py” with code 1 I put it there: $HOME/.config/rhasspy/profiles
Is next to profile.json

17

Can you provide a bit more log from Rhasspy?
The intent log in not really the interesting part, since it is correctly recognized

18

What kind of log do you need?
Where do I find it?

19

If you haven’t yet, @principe_mestizo, make sure to mark your handling script has executable:

$ chmod +x /path/to/handle.py

The handler scripts don’t have to be Python programs, so Rhasspy just executes them as a program. This means you have to have:

  1. The “shebang” line at the top of your script (#!/usr/bin/env python3)
  2. The script file marked as executable (chmod +x ...)
20

From the log button.
Surely the intent log you posted is not the only log right?

@synesthesiam gives good pointers as well, I never used a python script as intent handler

21

I followed all the steps, and make sure to place the exact path. Do you think the code is the problem or where do you think is?

22

@synesthesiam @romkabouter

When I change the WAV files to one of my local patch, do not play even though my local patch has chmod 777 and chmod + x

I do not know if this problem is related.

Note with WAV I mean:
${RHASSPY_BASE_DIR}/etc/wav/beep_hi.wav
${RHASSPY_BASE_DIR}/etc/wav/beep_lo.wav
${RHASSPY_BASE_DIR}/etc/wav/beep_error.wav

With:
/home/pi/Desktop/example.mp3

23

This folder:

is not known in the docker container.

Please post your log from Rhasspy, your whole log from where the intent is Recognized to the end of the session.
There might be an obvious error in it.

24

I have the 0.mp3 file with chmod 777 in the “profiles” folder and in “profiles/es”

In all WAV I put:
${RHASSPY_PROFILE_DIR}/0.mp3

It does not work, then I put:
propiles/0.mp3
and then
/propiles/0.mp3

Nothing works. It only works when I place the originals:

${RHASSPY_BASE_DIR}/etc/wav/beep_hi.wav
${RHASSPY_BASE_DIR}/etc/wav/beep_lo.wav
${RHASSPY_BASE_DIR}/etc/wav/beep_error.wav

With relation to the logs, I eliminate them (the blue button “CLEAR”), I can talk again, I copy them and send them, it does not matter, right?

25

Rhasspy only plays wav files, so no luck with mp3.

Correct about the logs, we need more verbose logs. Because there can be an error in it which causes your problem with the intent command

26

Perfect, I will generate the WAV.

With regard to the commands, you can show me an example that works for you? Please. One simple

27

The .wav has worked, but, the Handling command does not work.
Work: ${RHASSPY_PROFILE_DIR}/0.wav
It does not work: ${RHASSPY_PROFILE_DIR}/robot.py

Both have all the permissions.

The command starts with: #!/usr/bin/env python3

So I think the problem is something about the code, you can show me one that works, please? So I replenish it and see if it is that or another.

28

Again: please post the logs from Rhasspy.
If there is an error, it will be shown there.

30

Hmm that is a bit strange, maybe it is a setting to active more debug info.
I’ll have to check.

In my log there are a lot more message (about sending bytes to various topics)

31

Would it be too much to ask you to show me the example that works for you? Please. So I tell you the exact configuration and I rule out that it is a problem in my environment or not. Please.

32

This works for me (with handle.py in my profile directory [1]):

{
    "handle": {
        "command": {
            "program": "${RHASSPY_PROFILE_DIR}/handle.py"
        },
        "system": "command"
    }
}

[1] This is ~/.config/rhasspy/profiles/en on my machine

33

I do not use command and intent handler, so I have no example.
The example code you have should run fine.

34

I give up. Nothing works.

35

Can you show me what’s in your profile JSON? Did my example not work?

1 Like
36

I’ll try it out as well with the example and see what works.
You are using docker right?

1 Like
37

Yes, I am using Docker. The problem is that I think I have everything as you tell me. Because I changed the profile.json manually exactly as you told me, and I also tried when the web configurations make the change (there are no errors in this step, because it is programmed by you in Docker).

Above the python file I put what you told me so that it recognizes that it must be run with Python3 (#!/usr/bin/env python3), it has all the permissions (I made sure of that, all the folders before reaching the final path have 777 permissions), and I also made sure of which path is the correct one because it has the same format as the WAV of the wakeword (this works and is in the same path of the profile_dir… ${RHASSPY_PROFILE_DIR}/).

38

I am using the Home Assistant addon, so it will be a bit different for me :slight_smile:

Can you make screenshot of your Rhasspy settings? That might help

39

I have changed a coupe of lines in handle.py, my intent is “Locations” and I have added some line to print the chosen reply:

elif intent == "Locations":
    replies = ['Hi!', 'Hello!', 'Hey there!', 'Greetings.']
    reply = random.choice(replies) 
    print(reply)
    speech(reply)

This is the output.

image
image2010×216 44.6 KB

The file handle.py is set to ${RHASSPY_PROFILE_DIR}/handle.py

This is where the file is:

image
image1262×1234 128 KB

So if you set it to ${RHASSPY_PROFILE_DIR}/handle.py and put it in you active profile, see what happens.

please also attach the docker logs.: docker logs rhasspy

1 Like
40

Thanks to that log code I am seeing that it passes behind. I see that if you print the text you indicate, the problem is that it does not play the audio.

I have tried two things, in the first case what I did was to place the direct patch, and in 2nd case place the same as I put on the wakeup.

In the 1st case this comes out:

[DEBUG:2021-04-20 14:47:29,495] rhasspyremote_http_hermes: ['/profiles/es/handle.py']
[DEBUG:2021-04-20 14:47:29,503] rhasspywake_raven_hermes: Receiving audio
[DEBUG:2021-04-20 14:47:29,755] rhasspyremote_http_hermes: /home/pi/.config/rhasspy/profiles/es/wake.wav: No such file or directory
[WARNING:2021-04-20 14:47:29,756] rhasspyremote_http_hermes: Output: b'Hey there!\ntest-test-test\n'

…

cd /home/pi/.config/rhasspy/profiles/es/
ls
deepspeech          handle.py               profile.json  sentences.ini     supervisord.log  wake.wav

In the 2nd case this comes out:

[DEBUG:2021-04-20 14:42:19,892] rhasspyremote_http_hermes: Traceback (most recent call last):
  File "/profiles/es/handle.py", line 16, in <module>
    os.system(f"aplay ${RHASSPY_PROFILE_DIR}/wake.wav")
NameError: name 'RHASSPY_PROFILE_DIR' is not defined
[WARNING:2021-04-20 14:42:19,893] rhasspyremote_http_hermes: Output: b'Hi!\n'
[DEBUG:2021-04-20 14:42:21,450] rhasspy-wake-raven: Exiting refractory period
41

As I said, thanks to that logs command I can see that if it works (I did not know that command, I’m new in this world, so sorry for not seeing it before).

What does not work is the execution of the commands, I do not know why.

42

ok cool, so handly.py is working :slight_smile:
The ${RHASSPY_PROFILE_DIR} is not found in handle.py, because that variable is not known there.
Try it with
os.system(f"aplay /home/pi/.config/rhasspy/profiles/es/sounds/wake.wav")
Assuming you have that in your profile, please check the path.

What is it that you want to achieve? The original code:
speech(random.choice(replies))
should activate the Text to speech

Does this work in a terminal?
aplay /home/pi/.config/rhasspy/profiles/es/sounds/wake.wav

if not, the python script will certainly not work

43

In the log it does not work:

[DEBUG:2021-04-20 14:58:24,629] rhasspyremote_http_hermes: /home/pi/.config/rhasspy/profiles/es/wake.wav: No such file or directory

In Terminal works and reproduces well:

    aplay /home/pi/.config/rhasspy/profiles/es/wake.wav
    Playing WAVE '/home/pi/.config/rhasspy/profiles/es/wake.wav' : Signed 16 bit Little Endian, Rate 22050 Hz, Stereo

Is it honestly rare, there is some other way of playing audio? I will not use TTS.

44

Hmm ok, that is strange.

Can you post the whole python script?

45 
#!/usr/bin/env python3
import sys
import json
import os

data = json.loads(sys.stdin.read())
intent = data["intent"]["name"]
wake = data["wakeword_id"]

if wake:
    print("WORRRKKKKKKKKK!!!!!!!!!!!!!!!")
    print(wake)

if intent == "banal_1":
    os.system("aplay /home/pi/.config/rhasspy/profiles/es/wake.wav")
    print("test-test-test")
    print(intent)

Logs:

[DEBUG:2021-04-20 15:17:06,861] rhasspyremote_http_hermes: ['/profiles/es/handle.py']
[DEBUG:2021-04-20 15:17:07,091] rhasspyremote_http_hermes: /home/pi/.config/rhasspy/profiles/es/wake.wav: No such file or directory

[WARNING:2021-04-20 15:17:07,092] rhasspyremote_http_hermes: Failed to parse output as JSON: Expecting value: line 1 column 1 (char 0)
[WARNING:2021-04-20 15:17:07,092] rhasspyremote_http_hermes: Output: b'WORRRKKKKKKKKK!!!!!!!!!!!!!!!\noye_robot\ntest-test-test\nHola\n'
[DEBUG:2021-04-20 15:17:08,772] rhasspy-wake-raven: Exiting refractory period

Note “Hello” was the “Intent” I said. It registers everything well, executes the print correctly. The problem is sound.

46

I am trying more things, as with threads and does not work with the handle but if executing the direct python3, you can think of the reason?

import subprocess
    
def runcommand (cmd):
        proc = subprocess.Popen(cmd,
                                stdout=subprocess.PIPE,
                                stderr=subprocess.PIPE,
                                shell=True,
                                universal_newlines=True)
        std_out, std_err = proc.communicate()
        return proc.returncode, std_out, std_err

runcommand("aplay /home/pi/.config/rhasspy/profiles/es/wake.wav")

47

I do not know much about the os command, but you might want to try playsound

I do not know what you ar e trying to achieve, but for me this is clearly some python issue and not a Rhasspy issue :slight_smile:

1 Like
48

@romkabouter @synesthesiam A thousand thanks for all your help, really.

2 Likes
49

You could always POST the WAV data to Rhasspy itself:

curl -X POST \
     -H 'Content-Type: audio/wav' \
     --data-binary @/path/to/file.wav \
     'http://localhost:12101/api/play-wav'

or send an MQTT message:

mosquitto_pub -p 12183 \
              -t hermes/audioServer/default/playBytes/1 \
              -s < /path/to/file.wav
1 Like
51

It works on console, but not in the Python code of handle.py

[DEBUG:2021-04-20 18:05:55,459] rhasspyasr_deepspeech_hermes: Receiving audio
[DEBUG:2021-04-20 18:05:55,488] rhasspyremote_http_hermes: Warning: Couldn't read data from file "/home/pi/Desktop/wake.wav", this makes 
Warning: an empty POST.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    85  100    85    0     0   1024      0 --:--:-- --:--:-- --:--:--  1024
52

Log:

[DEBUG:2021-04-20 21:23:20,467] rhasspyremote_http_hermes: Traceback (most recent call last):
  File "/profiles/es/handle.py", line 5, in <module>
    from playsound import playsound
ModuleNotFoundError: No module named 'playsound'

The problem is that that comes out with other dependencies, most. I have them installed correctly, in console runs well.

53

I think with HTTP POST if it’s going to work, but I still can not make it work on Python. Can you help me?

I have this:

import requests

requests.post("http://localhost:12101/api/play-wav",
              data = "/home/pi/Desktop/demo.wav",
              headers = {"Content-Type": "audio/wav"})
54

Hmm ok, I don’t know. Maybe because the python is called from Rhasspy docker, which had playsound not installed.

1 Like
56

Do you know about this?

57

From what I have read in various topics, your biggest problem is not understanding how docker works. I assume you are trying to get the script to run as command script and therefore you are running inside docker and the path “/home/pi/Desktop/demo.wav” does not exist inside docker.

From the way it sounds when you write English is not your native language (it isn’t mine either) and most problems you face come from not recognizing what is available inside docker and what is outside. You should find yourself an introduction into docker in your language somewhere on the internet and visit the script problem after you know how to get files into docker, what is available inside docker and what is on the host system. That is not something, the rhasspy forum does cover naturally and while we try to help you, you seem to run into the same problem over and over when trying out different approaches to how to play sound from rhasspy.

You have stuff installec on the host system, and your commands work there, but the host system is completely different to the system rhasspy runs inside docker, so naturally everything you try in there doesn’t work if you try it the same way you do on the host system. The host system of course finds “/home/pi/Desktop/demo.wav”, but you can think of a docker container as a completely different linux system, and what you install on the host, or what files you have on the host, just is not available naturally in there. So you have to learn about how docker works and how to get stuff inside docker, about how the paths are different between host system and docker in regards to the profile folder ("/profiles/es" inside docker and “/home/pi/.config/rhasspy/profiles/es” on the host system). Without you having that understanding, there is not much we can help you with on this forum.

1 Like
58

You could say and only with that would have been enough.

But, all the other you said will appreciate it and take your advice. You’re right, but, up I’ve commented that I’m new on the subject, so I hope it has not been a lot of trouble.

I already solve the problem, I thank everyone for help (@Daenara @romkabouter @synesthesiam). :heart:

1 Like