Streaming online to Broadcastify with Liquid Soap

LiquidSoap can run multiple streams, maintain queues of wav, mp3, m4a etc files; https://www.liquidsoap.info/

(wav is all that is needed so set compressWav: false in TR v>4 as its not needed)

Code below now supports alpha tag uploading, taken from the talkgroupsFile csv setting.

Based off the following guide: Broadcastify Legacy via Liquidsoap

Install liquidsoap and get the service going.

The easiest way to do this is with liquidsoap docker image, as docker will then handle boot startup & logging rotation/limits

Docker-compose method

Create a yaml for docker-compose, with a container for each stream you run, here is an example of a 3 stream config:

# add '/var/run/liquidsoap:/var/run/liquidsoap' volume to your trunk recorder docker config, for them to share
version: '3.3'
services:
    stream0:
        user: root
        command:
            - /etc/liquidsoap/stream0.liq
        restart: always
        privileged: true        
        logging:
            options: 
                {'max-size':'1m', 'max-file':'5'}
        volumes:
            - '/root/radio/media:/app/media'
            - '/root/radio/streams:/etc/liquidsoap'
            - '/var/run/liquidsoap:/var/run/liquidsoap'
        image: 'savonet/liquidsoap:v2.0.3'
        container_name: stream0
    stream1:
        user: root
        command:
            - /etc/liquidsoap/stream1.liq
        restart: always
        privileged: true        
        logging:
            options: 
                {'max-size':'1m', 'max-file':'5'}
        volumes:
            - '/root/radio/media:/app/media'
            - '/root/radio/streams:/etc/liquidsoap'
            - '/var/run/liquidsoap:/var/run/liquidsoap'
        image: 'savonet/liquidsoap:v2.0.3'
        container_name: stream1

Note the command section for each stream, this is the liquidsoap config file that is run for each stream config below, and in this case located in /root/radio/streams on the host. You must create matching directory structure on local machine as specified in volume section to be mounted, in this case: /root/radio/media/ for recordings and /root/radio/streams/ to put the streamX.liq and stream.inc into

Add /var/run/liquidsoap:/var/run/liquidsoap volume to your trunk recorder docker config, for them to share.

To then start (once configs are setup below) use docker-compose -f liquidsoap.yaml up This will download the docker image, created the container, set it autostart and run it with the terminal attached. ctrl+c will stop the container but it will start automatically on next boot. added -d will start it detached to terminal in the background. as well as up, command there is stop, start and down

Stream config setup

I run multiple streams, so I put the common code in a file called stream.inc and then put the individual settings for each in a .liq file and include stream.inc from each. (according to .yaml above) these files go in /root/radio/streams/

Here is my stream.inc

#ok in docker and necessary as TR runs as root too
settings.init.allow_root.set(true)

# Configure Logging
log.stdout.set(true) #log to console, if running in docker, docker will save to file as well
log.file.set(false) 
log.file.path.set("<syslogdir>/<script>-#{STREAMID}.log")
log.level.set(2)

# create a socket to send commands to this instance of liquidsoap
settings.server.socket.set(true)
settings.server.socket.path.set("<sysrundir>/#{STREAMID}.sock")

# This creates a 1 second silence period generated programmatically (no disk reads)
silence = blank(duration=1.)

def append_title(meta) =
  [("title","Scanning...")]# Silence metadata
end

silence = map_metadata(append_title, silence)
# This creates a 1 second silence period generated programmatically (no disk reads)

recorder_queue = request.queue()

# If there is anything in the queue, play it.  If not, play the silence defined above repeatedly:
stream = fallback(track_sensitive=false, [recorder_queue, silence])

output.icecast( %mp3(stereo=false, bitrate=16, samplerate=22050),
  host=HOST, port=80, password=PASSWORD, genre="Scanner",
  description="Scanner audio", mount=MOUNT,  name=NAME, user="source", stream)

...and then stream0.liq, stream1.liq, stream2.liq, streamX.liq

#!/usr/bin/liquidsoap
HOST="audioX.broadcastify.com"
MOUNT="/mountPoint"
PASSWORD="password"
NAME="name-sent"
STREAMID="internal-name"
%include "stream.inc"

Change audioX under HOST to the audio server ID. audio1, audio2, etc.

Stream Use

Trunk recorder sound files are queued by using the uploadScript setting that is run at the completion of each recording per each system. Here is my python3 script that determines which stream to send a recording to and adds it to the specific liquid soap queue via a socket communication. Your logic for which stream to send it too will be based and your own setup and needs. Mine uses a txt file with one talkgroup id per line called TALKGROUP.list which acts as a whitelist to send those calls to STREAM0 from SYSTEM_NAME, and anything not in that list goes into STREAMID1. And everything from the other system I monitor goes into STREAM2. There is also some logic to determine if to delete the wav file or not, as during testing I keep some recordings for review later. Details about the recording passed to this script, from trunk recorder, are available in the JSON file data at jsonData. These can be used to work out which stream to send it to. Meta tags while possible are not used in this example, but maybe one day.

#!/usr/bin/python3
#
# for docker trunk recorder user root to run/execute this, set permissions on this file to 744
#
import socket
import sys
import os
import time
import re
import json
import shutil

delete = True #delete files after stream or not

filepath = sys.argv[1]
jsonpath = os.path.splitext(filepath)[0] + ".json"
#while tr v4+ does pass json path as arg, will still generate it this way for v3 compat

with open(jsonpath, 'r') as f:
  jsonData = json.load(f)
talkgroup = jsonData['talkgroup']


##############################
#archive unknown TG json/wav for later research, check /app/media/UNKNOWN/ exists, requires import shutil
#could just set delete = False, but its easier to check if all put in one folder
if jsonData['talkgroup_tag'] == '-':
    shutil.copy(filepath, '/app/media/UNKNOWN/'+ os.path.basename(filepath))
    jsonData['talkgroup_tag'] = str(talkgroup)+' Unknown'
##############################

# Create a UDS socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)

# Connect the socket to the port where the server is listening
# The following is example login to determine which stream to choose, customise as need

if 'SYSTEM_NAME' in filepath:
    with open('TALKGROUPS.list') as file:
        talkgroups= file.readlines()

    if talkgroup in talkgroups:
        server_address = '/var/run/liquidsoap/STREAMID0.sock'
    else:
        server_address = '/var/run/liquidsoap/STREAMID1.sock'
else:
    server_address = '/var/run/liquidsoap/STREAMID2.sock'

print ('connecting to %s' % server_address)

try:
        sock.connect(server_address)
except socket.error as msg:
        print('socket error')
        print(msg)
        sys.exit(1)

try:
    # Send data
    metaCommand = "annotate:title='"+jsonData['talkgroup_tag']+"':"
    if(delete):
        deleteCommand = "tmp:"
    
    message = 'request.queue_0.push {0}{1}{2}\n\r'.format(metaCommand,deleteCommand,filepath)
    #annotate: manually sets meta alpha tags from talkgroups csv and tmp: sets LS to delete after stream
    print('sending "%s"' % message.strip())
    sock.sendall(message.encode())

# useful for debugging by print liquidsoap socket response messages
#    while True:
#        data = sock.recv(16)
#        if (re.search('END', data.decode())):
#           break
#        print(data.decode())

finally:
    print('closing socket')
    quit = "exit\r\n"
    sock.send(quit.encode()) #to avoid disconnect without goodbye message, though not necessary :)
    time.sleep(0.5) #to avoid disconnect without goodbye message, though not necessary :)
    sock.close()
    os.remove(jsonpath)
    if delete:
        os.remove(jsonpath)

Note that once liquid soap is running a config and connected to broadcastify your stream will appear as online, regardless of whether trunk-recorder is running and sending anything to liquid soap or not.