spoa-server

Multi script langyage Stream Processing Offload Agent

This agent receive SPOP message and process it with script languages. The
language register callback with a message. Each callback receive the list
of arguments with types according with the language capabilities. The
callback write variables which are sent as response when the processing
is done.

Prerequirement

You have to install the development packages, either from the
distribution repositories or from the source.

CentOS/RHEL: sudo yum install python3-devel

The current minimal python version compatible with this library is 2.7.
It’s recommended to use python version 3 where possible due to python 2 deprecation.

Compilation

The server currently supports Lua and Python. Type “make” with the options:
USE_LUA=1 and/or USE_PYTHON=1.

You can add LUA_INC=… LUA_LIB=… to the make command to set the paths to
the lua header files and lua libraries.

Similarly, you can add PYTHON_INC=… PYTHON_LIB=… to the make command to set the paths to
the python header files and python libraries.
By default, it will try to compile by detecting the default python 3 parameters.
It will fall back to python 2 if python 3 is not available.

Start the service

After you have compiled it, to start the service, you just need to use “spoa”
binary:

$> ./spoa  -h
Usage: ./spoa [-h] [-d] [-p <port>] [-n <num-workers>]
    -h                  Print this message
    -d                  Enable the debug mode
    -p <port>           Specify the port to listen on (default: 12345)
    -n <num-workers>    Specify the number of workers (default: 5)
    -f <file>           Load script according with the supported languages

The file processor is recognized using the extension. .lua or .luac for lua and
.py for python. Start example:

$> ./spoa -d -f ps_lua.lua

 $> ./spoa -d -f ps_python.py

Configure

Sample configuration are join to this server:

spoa-server.conf : The HAProxy configuration file using SPOE server
spoa-server.spoe.conf : The SPOP description file used by HAProxy
ps_lua.lua : Processing Lua example
ps_python.py : Processing Python example

Considerations

This server is a beta version. It works fine, but some improvement will be
welcome:

Main process:

• Improve log management: Today the log are sent on stdout.
• Improve process management: The dead process are ignored.
• Implement systemd integration.
• Implement threads: It would be fine to implement thread working. Shared
  memory is welcome for managing database connection pool and something like
  that.
• Add PHP support and some other languages.

Python:

• Improve reporting: Catch python error message and report it in the right
  place. Today the error are dumped on stdout. How using syslog for logging
  stack traces ?

Maybe some other things…