Skip to content

Latest commit

 

History

History
965 lines (671 loc) · 49 KB

README.md

File metadata and controls

965 lines (671 loc) · 49 KB
Raw

ELM327-emulator

A Python emulator of the ELM327 OBD-II adapter connected to a vehicle.

ELM327-emulator provides an emulated OBD-II interface to client applications via TCP/IP networking, or serial communication (including pseudo-terminal function for non Windows operating systems), or direct interaction with communication devices and simulates an ELM327 adapter connected to a vehicle. It includes a command-line interface for extensive monitoring and controlling.

ELM327-emulator supports many operating systems including Windows, macOS and UNIX/Linux; it is agnostic of the client application and has been tested with python-OBD as well as with many applications on UNIX/Linux and on smartphone devices. It supports TCP/IP, Bluetooth, serial COM ports and pseudo-tty devices (for non Windows operating systems).

An internal dictionary (named ObdMessage) allows configuring the emulation, which is currently set to reproduce the message flow generated by a Toyota Auris Hybrid car including custom PIDs (through the scenario car option) and can be easily configured to statically and dynamically update its dictionary to simulate OBD-II answers produced by other vehicles. ELM327-emulator is able to support more protocols and the included dictionary uses the ISO 15765-4 CAN 11 bit ID 500 kbaud.

ELM327-emulator also includes a feature that compares direct connection with the OBD-II adapter to allow extending the dictionary by comparison with the emulated commands, as well as an auxiliary feature (obd_dictionary) that builds the PID dictionary of a specific vehicle by automatically querying all standard PIDs one by one (as well as querying additional custom PIDs specified by the user via CSV file); the dictionary can then be used to emulate the specific car.

Installation

Check that the Python version is 3.6 or higher (python3 -V), then install ELM327-emulator with the following command:

python3 -m pip install ELM327-emulator

Prerequisite components: pyyaml, python-daemon, obd; in addition, with Windows also tendo. All needed prerequisites are automatically installed with the package.

The following steps allow installing the latest version from GitHub.

  • Optional preliminary configuration with Ubuntu (if not already done):

    sudo apt-get update
sudo apt-get -y upgrade
sudo add-apt-repository universe # this is only needed if "sudo apt install python3-pip" fails
sudo apt-get update
sudo apt install -y python3-pip
python3 -m pip install --upgrade pip
sudo apt install -y git

  • Optional preliminary configuration with Windows:

    • install the latest version of Python
    • install git from Git-scm or using the Git for Windows installer
    • if the interface to use is a COM port, also install com0com (no installation is needed when using TCP/IP or Bluetooth interfaces)
    • check that PIP is upgraded (python3 -m pip install --upgrade pip)
    • Run this command:
    python3 -m pip install git+https://github.com/ircama/ELM327-emulator

Prerequisite components: pyyaml, python-daemon, obd; in addition, with Windows also tendo. All needed prerequisites are automatically installed with the package.

obd (python-OBD) is needed for obd_dictionary.

Important note: it is better to use an updated version of python-OBD package (e.g., the one installed from GitHub):

python3 -m pip install git+https://github.com/brendan-w/python-OBD.git

To uninstall:

python3 -m pip uninstall -y ELM327-emulator

Basic use

The emulator allows batch and interactive mode. The latter is the default and can be executed as follows:

python3 -m elm

or simply:

elm

After starting the program, the emulator is ready to use. To enable the preconfigured set of PIDs of a Toyota Auris Hybrid car, enter scenario car (or, alternatively, run the emulator with the -s car option, i.e python3 -m elm -s car).

By default ELM327-emulator uses the serial communication. The external application interfacing the emulator just needs to connect to the virtual device shown by the emulator and interact with the vehicle as if it was accessing a real ELM327 adapter.

Alternatively to the serial communication, ELM327-emulator supports TCP/IP networking through the -n option, followed by the port number (wich in most cases is 35000). Example:

python3 -m elm -n 35000

All subsequent information are not needed for a basic usage of the tool and allow mastering ELM327-emulator, exploiting it to test specific features including the simulation of communication exceptions, which are not always easy to be reproduced with a real link.

Compatibility

ELM327-emulator has been tested with Python 3.5, 3.6 and 3.7. Python 2 is not supported.

When using serial communication, with UNIX OSs, this code uses pty pseudo-terminals. With Windows, you should first install com0com (a kernel-mode virtual serial port driver), or other virtual serial port software; alternatively, cygwin and Windows Subsystem for Linux (WSL) are supported.

Running on Windows

When natively running on Windows (to be used when connecting a Windows application), ELM327-emulator requires a virtual serial port driver providing a virtual COM port pair (like com0com), so that one COM port (e.g., COM4) can be used to connect the application and the other one (e.g., COM3) the ELM327-emulator. By default, ELM327-emulator uses COM3 serial port; any other port can be set through the -p argument. Example:

python3 -m elm -p COM5

Usage

The description of the ELM327-emulator command-line option is the following:

usage: elm [-h] [-V] [-l] [-t] [-d] [-b FILE] [-p PORT] [-P DEVICE_PORT] [-a BAUDRATE] [-s SCENARIO]
           [-n INET_PORT] [-H INET_FORWARD_HOST] [-N INET_FORWARD_PORT] [-S FORWARD_SERIAL_PORT]
           [-B FORWARD_SERIAL_BAUDRATE] [-T FORWARD_TIMEOUT]

optional arguments:
  -h, --help            show this help message and exit
  -V, --version         Print ELM327-emulator version and exit
  -l, --newline         Use newline (<NL>) instead of carriage return <CR> for detecting a line separator
  -t, --terminate       Terminate the daemon process sending SIGTERM
  -d, --daemon          Run ELM327-emulator in daemon mode.
  -b FILE, --batch FILE
                        Run ELM327-emulator in batch mode. Argument is the output file. The first line in
                        that file will be the virtual serial device
  -p PORT, --port PORT  Set a serial communication port instead of using a pseudo-tty.
  -P DEVICE_PORT, --device DEVICE_PORT
                        Set the communication device to be opened instead of using a pseudo-tty port.
  -a BAUDRATE, --baudrate BAUDRATE
                        Set the serial device baud rate used by ELM327-emulator.
  -s SCENARIO, --scenario SCENARIO
                        Set the scenario used by ELM327-emulator.
  -n INET_PORT, --net INET_PORT
                        Set the INET socket port used by ELM327-emulator.
  -H INET_FORWARD_HOST, --forward_host INET_FORWARD_HOST
                        Set the INET host used by ELM327-emulator.when forwarding the client interaction
                        to a remote OBD-II port.
  -N INET_FORWARD_PORT, --forward_port INET_FORWARD_PORT
                        Set the INET socket port used by ELM327-emulator when forwarding the client
                        interaction to a remote OBD-II port.
  -S FORWARD_SERIAL_PORT, --forward_serial_port FORWARD_SERIAL_PORT
                        Set the serial device port used by ELM327-emulator when forwarding the client
                        interaction to a serial device.
  -B FORWARD_SERIAL_BAUDRATE, --forward_serial_baudrate FORWARD_SERIAL_BAUDRATE
                        Set the device baud rate used by ELM327-emulator when forwarding the client
                        interaction to a serial device.
  -T FORWARD_TIMEOUT, --forward_timeout FORWARD_TIMEOUT
                        Set forward timeout as floating number (default is 5 seconds).

ELM327-emulator v2.1.0 - ELM327 OBD-II adapter emulator

Description

The port to be used by the application interfacing the emulator is displayed when starting the program. E.g. on UNIX:

ELM327-emulator is running on /dev/pts/0

When running on Windows, the following message is shown:

ELM327-emulator is running on com0com serial port pair reading from COM3

Embedded dictionary of AT Commands and PIDs

A dictionary named ObdMessage is used to define commands and PIDs. The dictionary includes more sections (named scenarios):

  • 'AT': set of default AT commands
  • 'default': set of default PIDs
  • 'car': PIDs of a Toyota Auris Hybrid vehicle
  • any additional custom section can be used to define specific scenarios

Default settings include both the 'AT' and the 'default' scenarios.

The dictionary used to parse each ELM command is dynamically built as a union of three defined scenarios in the following order: 'default', 'AT', custom scenario (when applied). Each subsequent scenario redefines commands of the previous scenarios. In principle, 'AT' scenario is added to 'default' and, if a custom scenario is used, this is also added on top, and all equal keys are replaced. Then the Priority key defines the precedence to match elements.

If a custom scenario is selected through the scenario command, any key defined in the custom scenario replaces the default settings ('AT' and 'default' scenarios).

The key used in the dictionary consists of a unique identifier for each PID. Allowed values for each key (PID):

  • 'Request': received data; a regular expression can be used
  • 'Descr': string describing the PID
  • 'Exec': command to be executed
  • 'Log': logging.debug argument
  • 'ResponseFooter': run a function and returns a footer to the response (a lambda function can be used)
  • 'ResponseHeader': run a function and returns a header to the response (a lambda function can be used)
  • 'Response': returned data; can be a string, or a list/tuple of strings; if more strings are included, the emulator randomly select one of them each time
  • 'Action': can be set to 'skip' in order to skip the processing of the PID
  • 'Header': if set, process the command only if the corresponding header matches
  • 'Priority'=number: when set, the key has higher priority than the default (highest number = 1, lowest = 10 = default)

The emulator provides a monitoring front-end, supporting commands and controlling the backend thread which executes the actual process.

In 'ResponseFooter', 'ResponseHeader' and 'Response', spaces are stripped depending on configuration. Character ^ is substituted with space (and will not be stripped).

Built-in keywords

At the CMD> prompt, the emulator accepts the following commands:

  • help = List available commands (or detailed help with "help cmd").
  • port = Print the used TCP/IP port, or the used device, or the serial COM port, or the serial pseudo-tty, depending on the selected interface.
  • loglevel = If an argument is given, set the logging level, otherwise show the current one. Valid numbers: CRITICAL=50, ERROR=40, WARNING=30, INFO=20, DEBUG=10.
  • quit (or end-of-file/Control-D, or break/Control-C) = quit the program
  • counters = print the number of each executed PIDs (upper case names), the values associated to some 'AT' PIDs (cmd_...), the unknown requests, the emulator response delay, the total number of executed commands (commands) and the current scenario (scenario). The related dictionary is emulator.counters.
  • pause = pause the execution. (Related attribute is emulator.threadState = emulator.THREAD.PAUSED.)
  • prompt = toggle prompt off/on if no argument is used, or change the prompt if using an argument
  • resume = resume the execution after pausing; also prints the used device. (Related attribute is emulator.threadState = emulator.THREAD.ACTIVE)
  • delay <n> = delay each emulator response of <n> seconds (floating-point number; default is 0.5 seconds)
  • wait <n> = delay the execution of the next command of <n> seconds (floating-point number; default is 10 seconds)
  • engineoff = switch to engineoff scenario
  • scenario <scenario> = switch to <scenario> scenario; if the scenario is missing or invalid, defaults to 'car'. The autocompletion (by pressing or double-pressing TAB with UNIX systems) allows prompting all compatible scenarios defined in emulator.ObdMessage. (Related attribute is emulator.scenario.)
  • default = reset to default scenario
  • reset = reset the emulator (counters and variables)
  • color = toggle usage of colors off/on
  • history [<n>] = print the last 20 items of the command history; if an argument is given, print the last n items in the history; with argument clear, clears the history. The command history is permanently saved to file .ELM327_emulator_history within the home directory.
  • merge <module> = import a scenario from an external module and merges it with the emulator configuration. <module> shall be a Python file including the ObdMessage dictionary (e.g., generated by obd_dictionary) without .py extension (notice that the physical file shall be in the current directory and shall end with .py). The autocompletion with UNIX systems allows prompting all compatible files in the current directory (type merge, then press space, then press TAB). After a successful merge, the new scenario can be activated through the scenario command.
  • version = print the current version

In addition to the previously listed keywords, any Python command is allowed to query/configure the backend thread.

At the command prompt, cursors and keyboard shortcuts are allowed. Autocompletion (via TAB key) is active with UNIX systems for all previously described commands and also allows Python keywords and namespaces (built-ins, self and global). If the autocompletion matches a single item, this is immediately expanded; Conversely, if more possibilities are matched, none of them is returned, but pressing TAB again a list of available options is displayed. Tab autocompletion is not supported on Windows.

Special setters

The counters starting with cmd_... are special setters. They include cmd_echo, cmd_linefeeds, cmd_spaces, cmd_header, cmd_use_header, cmd_last_cmd.

echo and linefeed settings are both disabled by default. They can be configured via related AT commands (ATE1 and ATL1). The special setters cmd_echo and cmd_linefeeds allow enabling them via command line. Example:

emulator.counters['cmd_linefeeds'] = True; emulator.counters['cmd_echo'] = True

Possible values of emulator.counters['cmd_linefeeds']:

Value Behaviour Reference
0 (Default) Each line is closed by two CRs. \r\r
1 Each line is closed by two CRs and a LF. \r\r\n
2 Each line is closed by a CR and a LF. \r\n
3 Each line is closed by a LF. \n
4 Each line is closed by a CR. \r

Space characters are inserted by default in the ECU response as per specification. To remove them, use the AT command ATS0 or emulator.counters['cmd_spaces'] = 0.

By default the header is not included in the ECU response. To add it, use the AT command ATH1 or emulator.counters['cmd_use_header'] = True.

The default ECU header is ECU_ADDR_E (e.g., "7E0", producing answer "7E8"; ref. obd_message.py). Use cmd_header to customize it.

The last executed command is stored in cmd_last_cmd. This is used to repeat the command when the 'fast' option is set (command repetition, through a newline).

Each time the interface is reset by an ATZ command, the special setters are restored to their default settings and any specific customization needs to be issued again. Use emulator.presets in order to preset the special setters so that they are applied as default values each time the interface is opened by an application. Example:

emulator.presets = { 'cmd_linefeeds': 4, 'cmd_spaces': 0 }

Advanced usage

The emulator includes a timeout management for each entered character, which by default is not active (e.g., set to 1440 seconds). This setting can be configured through emulator.counters['req_timeout']. Decimals are allowed. Some adapters provide a feature that discards characters if each of them is not entered within a short time limit (apart from the first one after a CR/Carriage Return). The appropriate emulation for this timeout is to set emulator.counters['req_timeout']=0.015 (e.g., 15 milliseconds). Typing commands by hand via terminal emulator with such adapters is not possible as the allowed timing is too short. The same happens when setting req_timeout to 0.015.

The command prompt also allows configuring the emulator.answer dictionary, which has the goal to dynamically redefine answers for specific PIDs ('Pid': '...'). Its syntax is:

emulator.answer = { 'pid' : 'answer', 'pid' : 'answer', ... }

Example:

emulator.answer = { 'SPEED': 'NO DATA\r', 'RPM': 'NO DATA\r' }
# Or, alternatively:
emulator.answer['SPEED']='NO DATA\r'
emulator.answer['RPM']='NO DATA\r'

The above example forces SPEED and RPM PIDs to always return "NO DATA".

To reset the emulator.answer string to its default value:

emulator.answer = {}
# Or, alternatively:
del emulator.answer['SPEED']
del emulator.answer['RPM']

To simulate that the adapter is not connected to the vehicle:

emulator.answer['AT_R_VOLT'] = '0.0V'

The emulator.ELM_R_UNKNOWN parameter allows customizing the message returned in case of unknown/invalid command. The default message is ?\r, with an addition of a trailing \r. This message can be customized; for example, to just get \r, set the following:

emulator.ELM_R_UNKNOWN = ''

The dictionary can be used to modify answers within a workflow. The front-end allows implementing basic Python workflows and, when used in batch mode, can also be controlled by a piped external supervisor. The following examples show some simple workflows in interactive mode.

Example of automation which suspends the emulator for 10 seconds:

emulator.threadState = emulator.THREAD.PAUSED; time.sleep(10); emulator.threadState = emulator.THREAD.ACTIVE

Example of an automation that simulates the off/on ignition states:

CMD> for i in range(10): emulator.scenario="car" if i % 2 else "engineoff"; print(emulator.scenario); time.sleep(10)
engineoff
car
engineoff
car
engineoff
car
engineoff
car
engineoff
car

Configuring response strings

Response strings allow embedding Python statements and expressions. Specifically, Response, ResponseHeader, ResponseFooter, emulator.answer and emulator.ELM_R_UNKNOWN support single and multiple in-line Python commands (expressions or statements) when embraced between \0 tags: this feature for instance can be used to embed real-time delays between strings or to differentiate answers. The return value of a statement is ignored. The evaluation of an expression is substituted. Spaces inside \0 tags are allowed and can be used to improve readability. Example: 'Response' = 'SEARCHING...\0 time.sleep(1) \0\rUNABLE TO CONNECT\r'. This returns SEARCHING..., then waits one second, then returns \rUNABLE TO CONNECT\r. Notice that, as time.sleep is a statement, the related return value is ignored.

Further processing can be achieved through a lambda function applied to ResponseHeader, ResponseFooter. It has to manage the following parameters: self, cmd, pid, val (e.g., lambda self, cmd, pid, val:).

  • cmd: the request, received by the client application
  • pid: the PID identifier (which can be used as key to index self.counters and ObdMessage)
  • val: ObdMessage value related to pid (e.g., val['Response']).

Example of PID definition within the ObdMessage dictionary:

            'ELM_PIDS_A': {
                'Request': '^0100$',
                'Descr': 'PIDS_A',
                'ResponseHeader': \
                lambda self, cmd, pid, val: \
                    'SEARCHING...\0 time.sleep(1) \0\rUNABLE TO CONNECT\r' \
                    if self.counters[pid] == 1 else 'NO DATA\r',
                'Response': '',
                'Priority': 5
            },

In the above example, the first time ResponseHeader is executed, the produced response is SEARCHING..., followed by a one-second delay and then \rUNABLE TO CONNECT\r. For all subsequent messages, the response will be different and produces 'NO DATA\r'.

The ability to add dynamic differentiators and delays within responses enables testing specific use cases and exceptions that are difficult to be achieved through a real connection with a car. These not only apply to the ObdMessage dictionary (by editing obd_message.py), but also to emulator.answer and emulator.ELM_R_UNKNOWN, that can be configured through the command line. Consider for instance the following dynamic configuration via command line:

emulator.answer['SPEED'] = '\0 ECU_R_ADDR_E + " 03 41 0D 0A " if randint(0, 100) > 20 else "NO DATA" \0\r'

In the above example, which illustrates an in-line expression substitution, the configuration of the ‘SPEED’ PID (Vehicle speed) is replaced with a dynamic answer and the ‘SPEED’ PID will return 7E8 03 41 0D 0A + newline for most of the time. With 20% probability, NO DATA + newline is returned. Notice that the last \r is common to both options.

The following example shows how to dynamically generate an answer via command line by converting decimal numbers to hex string in order to allow comfortable testing of a PID by specifying decimal input values. Suppose that the PID needs to double the input. We use CUSTOM_FUEL_LEVEL PID in the example, testing the answer related to 15.5 liters.

Preliminarily, test number conversion with the command line:

"%.2X" % int(15.5*2)
1F

Apply it to CUSTOM_FUEL_LEVEL PID so that it returns 7C0 03 61 29 1F \r':

emulator.answer['CUSTOM_FUEL_LEVEL'] = '7C0 03 61 29 \0 "%.2X" % int(15.5*2) \0 \r'

Or, alternatively, use the header variable instead of the header digits:

emulator.answer['CUSTOM_FUEL_LEVEL'] = '\0 ECU_R_ADDR_I + " 03 61 29 " + "%.2X" % int(15.5*2) \0 \r'

The following command sets SPEED (Vehicle speed) to 60 km/h via command line (60 can be changed to any integer value between 0 and 255):

emulator.answer['SPEED'] = '\0 ECU_R_ADDR_E + " 03 41 0D %.2X" % 60 \0\r'

The following command sets RPM (Engine RPM) to 500 via command line:

emulator.answer['RPM'] = '\0 ECU_R_ADDR_E + " 04 41 0C %.4X" % int(4 * 500) \0\r'

To list the configuration, type emulator.answer, or simply counters. To remove the dynamic answer and return to the default configuration of the ‘SPEED’ PID, type del emulator.answer['SPEED'].

Command to configure PID '0100' answer (PIDS_A) to BUS INIT: OK for its first query and to 48 6B 13 41 00 BE 1F B8 11 AD \r for all the subsequent queries:

emulator.answer['ELM_PIDS_A'] = '\0 "BUS INIT: OK" if self.counters["ELM_PIDS_A"] < 2 else "48 6B 13 41 00 BE 1F B8 11 AD \\r" \0'

The ELM_PIDS_A counter (emulator.counters["ELM_PIDS_A"]) can be reset with:

emulator.counters["ELM_PIDS_A"] = 0

Available interfaces

ELM327-emulator allows the following interfaces:

  • serial communication using a pseudo-terminal, as default mode on non Windows operating systems (without options),
  • TCP/IP networking, when using option -n, followed by the TCP/IP port,
  • serial COM port, when using option -p, default mode with Windows (the option is followed by the port name and allows setting a baud rate with the -a option),
  • standard communication, when using option -P.

Usage of a pseudo-terminal

On non Windows operating systems, the default mode (without communication options) creates a pseudo-tty driver to be used to connect client applications. It is shown at startup, can be read by applications in batch mode and the port command returns it at any moment.

python3 -m elm -s car

Usage of a TCP/IP connection

The -n options uses a TCP socket; the most commonly used one is 35000.

python3 -m elm -s car -n 35000

Usage of a serial communication port

The -p option exploits the pyserial library. A baud rate can be set with the -a option (defaulted to 38400 bps). With Windows, the COM port is defaulted to COM3.

To open a specific Windows port:

python3 -m elm -s car -p COM3

To open a specific Windows port (which is the same as python3 -m elm -s car because corresponds to default values):

python3 -m elm -s car -p COM3 -a 38400

Usage of the standard communication option

The -P option uses the standard communication mode allowed by the operating system when opening a read/write device, including a special file (for instance, for Bluetooth serial networking with UNIX/Linux).

Usage of Bluetooth with Windows

We need to pair the client device and add an incoming RFCOMM port. Then ELM327-emulator can access that virtual COM port with the -p option.

With Windows, run "Settings", "Bluetooth & other devices", "More Bluetooth options", "COM ports" tab, press "Add", select "Incoming", press "OK". For instance, this procedure creates a virtual COM4 port.

Then run:

python -m elm -p COM4 -s car

Note: adding the -l option may be needed in some cases and is important when the configuration changes the way command lines are sent, closing lines with a newline instead of a CR.

Usage of Bluetooth with UNIX

After creating a Bluetooth special file on a UNIX system implementing an RFCOMM port, ELM327-emulator can access that file with the -P option.

Install Bluetooth tools and daemons:

sudo apt-get install bluez

Pairing the devices:

bluetoothctl
[bluetooth]# power on
[bluetooth]# agent on
[bluetooth]# default-agent
[bluetooth]# discoverable on
[bluetooth]# pairable on

Run the client application, select the Bluetooth device name, then perform pairing on both systems, answering yes.

Note. In case removing or unpairing a paired address is needed:

remove <MAC address>
power off

After pairing, quit bluetoothctl:

[bluetooth]# exit

Bluetooth preparation:

sudo service bluetooth restart
sdptool add --channel=1 SP
rfcomm release 0 # if needed

Configure the special file /dev/rfcomm0 (see below):

sudo mknod -m 666 /dev/rfcomm0 c 216 0
sudo chown $USER /dev/rfcomm0

Run ELM327-emulator with bluetooth interface:

rfcomm watch /dev/rfcomm0 1 python3 -m elm -P /dev/rfcomm0 -l -s car

Note: the -l option may be needed in some cases and is important when the Bluetooth configuration changes the way command lines are sent, closing lines with a newline instead of a CR.

The /dev/rfcomm0 device driver can be manually created in order to avoid the error "Can't open RFCOMM device: Permission denied" when running rfcomm as a standard user (an not as root). First run rfcomm as superuser and connect a client:

sudo rm -f /dev/rfcomm0; sudo rfcomm listen /dev/rfcomm0 1 ls -l /dev/rfcomm0

After a client connection, you should get:

crw-rw---- 1 root dialout 216, 0 mar 28 11:17 /dev/rfcomm0

Create the special file with the same major_number and minor_number, using 666 permissions and change ownership:

sudo mknod -m 666 /dev/rfcomm0 c 216 0
sudo chown $USER /dev/rfcomm0

Note: the Bluetooth error "Can't bind RFCOMM socket: Address already in use" means that the bind() function used by the command producing the error failed because there is another socket with the same number already bound by a local application. The way to solve this problem is to find the local application binding that socket and terminating it.

Forwarder options

In order to verify and improve its dictionary, ELM327-emulator allows acting as a proxy between a software application and a real OBD-II device. Whenever the OBD-II interface provides data to the application which the dictionary does not include, a warning is shown reporting the answer from the OBD-II interface; besides, a related unknown_<command>_R element is added to the counters, with the last returned answer from the OBD-II interface, to allow subsequent verifications. The dictionary can then be manually edited to align the ELM327-emulator behaviour with the answer returned by the OBD-II interface.

To act as a proxy, ELM327-emulator can either expose the virtual serial device or the TCP network port to the application; then it connects the OBD-II interface through an internal forwarder component, allowing serial communication or TCP/IP networking.

The selection of the interface exposed to the application is done via the standard options: by default the pseudo-tty device is used; alternatively, the -n option allows using a local TCP/IP port (e.g., -n 35000).

The OBD-II interface is connected through the -S option (serial device) or the -H and -N ones (TCP/IP host and related port). When using the serial device, the -B option allows indicating a specific baud rate (38400 bps by default).

Data read from the OBD-II port are grouped together basing on a timeout parameter (floating point number) which by default is 5.0 seconds and can be tuned with the -T option. The higher the number, the more reliant the grouping; anyway, delays produced by high timeout values might compromise the communication quality: if the application does not perform correctly in the forwarder mode (e.g., producing connection drops), is useful to test lower timeout periods, like -T 0.2 or -T 0.1.

Example.

  • In a window, run a simulated OBD-II interface connected via TCP network: python3 -m elm -s car -n 20000. Then optionally set loglevel 10.
  • In another window, run ELM327-emulator configured as forwarder to the local TCP port 20000 and exposing a network port 35000: python3 -m elm -s car -n 35000 -N 20000 -H localhost -T 0.2.
  • In a third window, run a telnet client: elnet localhost 35000. Write at@1 and press enter. Check the logs in the other windows.
  • Close the telnet client. Run OBD Auto Doctor, select Wifi communication, IP address 127.0.0.1, port 35000. Check the logs in the other windows. You should succeed in connecting the application.

Logging and monitoring

Logs are written to elm.log, file, rotated to elm.log.1 and elm.log.2 when its size reaches 1 MB. Logging is controlled through the elm.yaml file (in the current directory by default). Its path can be set through the ELM_LOG_CFG environment variable. This file follows the Python’s builtin logging module format and allows customizing the configuration of the logging process.

The logging level can be dynamically changed through the loglevel command.

To read the current log level:

loglevel

To set log level to debug:

loglevel 10

Alternatively, the logging level can be set through logging.getLogger().handlers[n].setLevel(). To check that console is the first handler (e.g., handlers[0]), run for n, l in enumerate(logging.getLogger().handlers): print(n, l.name). For instance, if console refers to the first handler (default settings of the provided elm.yaml file), the following commands will change the logging level:

logging.getLogger().handlers[0].setLevel(logging.DEBUG)
logging.getLogger().handlers[0].setLevel(logging.INFO)
logging.getLogger().handlers[0].setLevel(logging.WARNING)
logging.getLogger().handlers[0].setLevel(logging.ERROR)
logging.getLogger().handlers[0].setLevel(logging.CRITICAL)

It is possible to add marks in the log file via commands like logging.info("my mark")

To totally disable logging for all handlers: logging.disable(logging.CRITICAL). To restore logging: logging.disable(0).

Command to count the number of different PIDs (OBD Commands) used by the client (excluding AT Commands):

import re
from functools import reduce
reduce(lambda x, key: x + (1 if re.match('^[A-Z]', key) and not key.startswith('AT_') and emulator.counters[key] > 0 else 0), emulator.counters, 0)

The following command returns the total number of OBD Commands (PID queries issued by the client excluding AT Commands):

import re
from functools import reduce
reduce(lambda x, key: x + (emulator.counters[key] if re.match('^[A-Z]', key) and not key.startswith('AT_') else 0), emulator.counters, 0)

To only count AT Commands:

from functools import reduce
reduce(lambda x, key: x + (emulator.counters[key] if key.startswith('AT_') else 0), emulator.counters, 0)

Print the average number of processed commands per second within a 5 seconds period:

a=emulator.counters['commands'];time.sleep(5);print((emulator.counters['commands']-a)/5)

To save a CSV file including the emulator.counters dictionary:

with open('mycounters.csv', 'w') as f: f.write('\r\n'.join([x + ', ' + repr(emulator.counters[x]) for x in emulator.counters]))

ObdMessage Dictionary Generator for "ELM327-emulator" (obd_dictionary)

obd_dictionary is a dictionary generator for "ELM327-emulator".

It queries the vehicle via python-OBD for all available commands and is also able to process custom PIDs described in Torque CSV files.

Its output is a Python ObdMessage dictionary that can either replace the obd_message.py module of ELM327-emulator, or extend the existing dictionary via merge command, so that the emulator will be able to provide the same commands returned by the vehicle.

Notice that querying the vehicle might be invasive and some commands can change the car configuration (enabling or disabling belts alarm, enabling or disabling reverse beeps, clearing diagnostic codes, controlling fans, etc.). In order to prevent dangerous PIDs to be used for building the dictionary, a PID blacklist (blacklisted_pids) can be edited in elm.py. To check all PIDs without performing actual OBD-II queries (dry-run mode), use the -p 0 option (the standard error output with default logging level shows the list of produced PIDs).

obd_dictionary can be run as:

python3 -m obd_dictionary --help

or simply:

obd_dictionary --help

Command line arguments:

usage: obd_dictionary [-h] -i DEVICE [-c CSV_FILE] [-o FILE] [-v] [-V] [-p PROBES] [-B BAUDRATE]
                      [-T TIMEOUT] [-C] [-F] [-P PROTOCOL] [-d DELAY] [-D DELAY_COMMANDS] [-n CAR_NAME]
                      [-b] [-r] [-x] [-t [FILE]] [-m]

optional arguments:
  -h, --help            show this help message and exit
  -i DEVICE             python-OBD interface: serial port connected to the ELM327 adapter (required
                        argument).
  -c CSV_FILE, --csv CSV_FILE
                        input csv file including custom PIDs (Torque CSV Format: https://torque-
                        bhp.com/wiki/PIDs) '-' reads data from the standard input
  -o FILE, --out FILE   output dictionary file generated after processing input data (replaced if
                        existing). Default is to print data to the standard output
  -v, --verbosity       print process information
  -V, --verbosity_debug
                        print debug information
  -p PROBES, --probes PROBES
                        number of probes (each probe includes querying all PIDs to the OBD-II adapter)
  -B BAUDRATE, --baudrate BAUDRATE
                        python-OBD interface: baudrate at which to set the serial connection.
  -T TIMEOUT, --timeout TIMEOUT
                        python-OBD interface: specifies the connection timeout in seconds.
  -C, --no_check_voltage
                        python-OBD interface: skip detection of the car supply voltage.
  -F, --fast            python-OBD interface: allows command optimization (CR to repeat, response limit).
  -P PROTOCOL, --protocol PROTOCOL
                        python-OBD interface: forces using the given protocol when communicating with the
                        adapter.
  -d DELAY, --delay DELAY
                        delay (in seconds) between probes
  -D DELAY_COMMANDS, --delay_commands DELAY_COMMANDS
                        delay (in seconds) between each PID query within all probes
  -n CAR_NAME, --name CAR_NAME
                        name of the car (dictionary label; default is "car")
  -b, --blacklist       include blacklisted PIDs within probes
  -r, --dry-run         test the python-OBD interface in debug mode.
  -x, --noautopid       do not autopopulate the pid list with the set of built-in commands supported by
                        the vehicle; only use csv file.
  -t [FILE], --at [FILE]
                        include AT Commands within probes. If a dictionary file is given, also extract AT
                        Commands from the input file and add them to the output
  -m, --missing         add in-line comment to dictionary for PIDs with missing response

ObdMessage Dictionary Generator for "ELM327-emulator".

Sample usage: obd_dictionary -i /dev/ttyUSB0 -c car.csv -o AurisOutput.py -v -p 10 -d 1 -n mycar

obd_dictionary exploits the command discovery feature of python-OBD, which autopopulates the set of builtin commands supported by the vehicle through queries performed within the connection phase. Optionally, this set can be further enriched with a list of custom PIDs included in an input csv file in Torque CSV Format. The autopopulation feature can be disabled with -x option.

The command allows all the python-OBD interface settings (see -B, -T, -C, -F, -P command-line options) and a dry-run flag (-r), which is very useful to test the OBD-II connection.

For instance, the following command tests an OBD-II connection via Bluetooth using the related recommendations described in the python-OBD repository.

python3 -m obd_dictionary -i /dev/rfcomm0 -B 38400 -T 30 -r

See also this post for Bluetooth.

For better analysis, the -r output can be piped to lnav (the following command tests the USB connection):

python3 -m obd_dictionary -i /dev/ttyUSB0 -B 38400 -r 2>&1 | lnav

When the tests provide successful connection, the -r option can be removed and the additional obd_dictionary options can be added.

obd_dictionary can be also used to test elm. Run python3 -m elm -s car. Read the pseudo-tty, say /dev/pts/2 (this mode uses the serial communication). Run obd_dictionary:

python3 -m obd_dictionary -i /dev/pts/2 2>&1 | lnav

(The automation of this process is shown further on.)

In general, ELM327-emulator should already manage all needed AT Commands within its default dictionary, so in most cases it is worthwhile removing them from the new scenario via -t option.

The file produced by obd_dictionary provides the same information model of obd_message.py. It can be used to replace the default module or can be dynamically imported in ELM327-emulator through the merge command, which loads an ObdMessage dictionary and merges it to emulator.ObdMessage. Example of merge process:

# Create AurisOutput.py
obd_dictionary -i /dev/ttyUSB0 -c auris.csv -o AurisOutput.py -n Auris
python3 -m elm # run ELM327-emulator
merge AurisOutput
scenario Auris

To help to configure the emulator, autocompletion is allowed (by pressing TAB) with UNIX systems when prompting the merge command, including the merge argument. Also variables and keywords like scenario accept autocompletion, including the scenario argument.

A merged scenario can be removed via del emulator.ObdMessage['<name of the scenario to be removed>'].

To produce a complete dictionary file that can replace obd_dictionary:

obd_dictionary -i /dev/ttyUSB0 -c auris.csv -o AurisOutput.py -n default -t elm/obd_message.py

ELM327-emulator batch mode

ELM327-emulator can be run in batch mode to allow automating tests and background execution. The -b FILE option allows this mode and writes the output to FILE. When using serial communication, the first line in that file will be the virtual serial device, which can be read to a shell variable through read variable_name < output_file. Commands can be piped in (e.g., within a bash script) to configure the emulator (e.g., via echo -e). The appropriate way to kill a background instance of the emulator is with the SIGINT signal (kill -2). To ensure that the external application is started only after correct setup of the emulator, the input commands can be terminated with a string (e.g., "RUNNING") that can then be recognised before starting the application.

elm offers four operation modes:

  • interactive (providing a command line prompt). This is activated by default, when neither -b option nor -d is used;
  • batch mode with input commands, activated when the -b option (-b file or -b -e.g., -b - for standard log output, or -b output_file_name). This mode reads the standard input for the same commands that can be issued by the user in interactive mode;
  • batch mode light, only available with UNIX, without input commands (no separate thread is created and the command interpreter is not used). This is activated when both -b and -d options are used;
  • daemon mode without input commands, only available with UNIX, allowing starting and stop the process in UNIX daemon mode, with -d option (start) and -t (terminate).

In daemon mode, a lock file is used to prevent multiple instances.

Note: with Windows, options -d and -t are not available.

The following script shows an example of batch mode usage. obd_dictionary is run after starting ELM327-emulator in background and is used here as example of external application interfacing the emulator. The output of the emulator is saved to $FILE and the background process id is saved to $EMUL_PID.

FILE=/tmp/elm$$
echo -e 'scenario car\ncounters' | elm -b "${FILE}" &
EMUL_PID=$!

until [ ! -f "${FILE}" ] || grep -q "End of batch commands." "${FILE}"
do sleep 0.5
done
read TTYNAME < "${FILE}"

obd_dictionary -i "${TTYNAME}" -t -v -o /dev/null

kill -INT "${EMUL_PID}"
cat "${FILE}"
rm "${FILE}"

Python API

Instantiating the class

All arguments are optional.

from elm import Elm

emulator = Elm(
    batch_mode=False,           # optional flag to indicate different logging for batch mode
    newline=False,              # optional flag to use newline (<NL>) instead of carriage return <CR> for detecting a line separator
    serial_port="",             # optional serial port used with Windows (ignored with non Windows O.S.)
    serial_baudrate="",         # baud rate used by any serial port (but the forward port); default is 38400 bps
    net_port=None,              # number for the optional TCP/IP network port, alternative to serial_port
    forward_net_host=None,      # host used when forwarding the client interaction to a remote OBD-II device
    forward_net_port=None,      # port used when forwarding the client interaction to a remote OBD-II device
    forward_serial_port=None,   # serial port name when forwarding the client interaction to an OBD-II device via serial communication
    forward_serial_baudrate = None, # used baud rate for the forwarded serial port; default is 38400 bps
    forward_timeout=None)       # floating point number indicating the read timeout when configuring a forwarded OBD-II device; default is 5.0 secs.

get_pty() returns the used port.

Interactive mode

Interactive mode uses the Context Manager:

from elm import Elm
import time

with Elm() as session:
    # interactive monitoring
    pty = session.get_pty()
    print(f"Used port: {pty}")
    time.sleep(40) # example

Example of TCP/IP network usage:

from elm import Elm
import time

with Elm(net_port=35000) as session:
    time.sleep(40) # example

Batch/daemon mode

Batch mode without interaction does not need the Context Manager:

from elm import Elm

emulator = Elm(batch_mode=True)

pty = emulator.get_pty()
print(f"Used port: {pty}")

emulator.run()

Software architecture

When using the Context Manager, a thread is started and the current context is returned to the user. The created thread opens a bidirectional pty-type pipe and processes the related I/O.

When not using the Context Manager, no background thread is created and the pipe is run in the current context.

Testing OBD-II applications

Simple testing

With UNIX, the serial communication can be tested with screen.

python3 -m elm

In another terminal:

sudo apt-get install screen
screen /dev/pts/3

The TCP/IP networking can be tested via telnet.

python3 -m elm -n 35000

In another terminal:

sudo apt-get install telnet
telnet localhost 35000

OBD Auto Doctor

One of the most useful applications which can be used to test ELM327-emulator is OBD Auto Doctor. It supports different operating systems, including Windows, Mac, Linux, Android, iOS and enables communicating with OBD-II to get summary information, trouble codes, advanced diagnostics, real time graphical monitoring and many other in-depth data on the EQUs.

This application allows a demo license which can be used for testing of ELM327-emulator. The following instructions explain how to do this with Ubuntu:

Download the Linux application from the Obdautodoctor site. Check prerequisites.

Install with sudo dpkg -i obd-auto-doctor....deb.

Run ELM327-emulator and select the car scenario:

python -m elm -s car

This mode uses the serial communication. Copy the pseudo-tty device reported by ELM327-emulator.

Run OBD Auto Doctor with obdautodoctor. Then select File, Open connection, Connection method: Serial port. Use manual settings. Paste the pseudo-tty device in the COM port box. Press Connect.

OBD Auto Doctor also supports TCP/IP communication. Run ELM327-emulator using TCP/IP networking:

python -m elm -s car -n 35000

Run OBD Auto Doctor with obdautodoctor. Then select File, Open connection, Connection method: WiFi. IP address: 127.0.0.1. Port: 35000. Press Connect.

scantool

Scantool from ScanTool.net is an old software which can also be used to test ELM327-emulator.

Recent repository: https://github.com/kees/scantool

Software ported to Ubuntu 20.04 LTS: https://github.com/ircama/scantool/tree/pts_support

Installation:

git clone --branch pts_support https://github.com/Ircama/scantool.git
sudo apt install liballegro4.4 liballegro4-dev allegro4-doc
make clean
make -e RELEASE=yes LOG=yes

To run the application: ./scantool

This version of scantool allows configuring a pseudo-tty support by editing ~/.scantoolrc. See related readme.txt. With Ubuntu, this can be automated by ELM327-emulator via the following plugin:

import fileinput
import os
import re

def scantool(port):
    numport = str(int(os.path.basename(port)) + 1000)
    with fileinput.FileInput(os.path.expanduser("~/.scantoolrc"), inplace=True) as file:
        for line in file:
            match = re.sub(r"^comport_number *=.*", "comport_number = " + numport, line)
            if match:
                print(match, end='')
            else:
                print(line, end='')

Close scantool, save a file named scantool.py including the above reported plugin. Run ELM327-emulator:

python3 -m elm
scenario car
from scantool import scantool;scantool(emulator.slave_name) # load and run the plugin

To run the application integrated with ELM327-emulator: ./scantool

License

(C) Ircama 2021 - CC BY-NC-SA 4.0

Credits

Many thanks to @qqj1228 for implementing support to com0com Windows driver as well as for other enhancements.