Well...
Since ModBus is a specification, here is what they say:
RTU Mode
1. 8 Bit Binary Coding System
2. 1 Start Bit
3. 8 Data bits, LSB sent first
4. 1 Bit for Parity (Devices are REQUIRED to support EVEN parity)
5. 1 Stop bit
In RTU mode, messages end with 2 bytes which is a 16 bit CRC. All devices are required to support RTU mode as the default sending mode, so when auto-checking for what mode the device is sending in, you should start with trying to detect RTU mode.
ASCII Mode
1. Hexadecimal ASCII characters 0-9 and A-F Coding (sent as pairs)
2. 1 Start Bit
3. 7 Data bits, LSB first
4. 1 Parity Bit (Devices are REQUIRED to support EVEN parity)
5. 1 Stop Bit
In ASCII mode, messages must start with 0x3A (a colon) and end with CRLF (0D0A).
9600bps and 19,200bps are required, 19,200bps is the default network speed.
So what I would do if I had to auto-negotiate a network is:
1. Select RTU mode in my driver
2. Set my Serial port to 19200,8,E,1
3. May have to send a broadcast message here to get the devices to talk
4. Listen to the incoming message
5. Check to see if it passes the CRC, if not, change the serial port settings
6. Continue between step 2 and 5 for all baud rates, parities
7. If they all fail, switch to ASCII mode at step 1 and continue from step 2
Link to the ModBus Serial Specification[
^]