README
The
REFCODES.ORG
codes represent a group of artifacts consolidating parts of my work in the past years. Several topics are covered which I consider useful for you, programmers, developers and software engineers.
What is this repository for?
This toolkit targets at programming serial communication at a high abstraction level, addressing issues such endianess, length allocations, CRC
checksums, acknowledgement management or handshake processing whilst keeping you in full control over your bits and bytes.
How do I get set up?
To get up and running, include the following dependency (without the three dots “…”) in your pom.xml
:
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-serial</artifactId>
<groupId>org.refcodes</groupId>
<version>3.3.9</version>
</dependency>
...
</dependencies>
The artifact is hosted directly at Maven Central. Jump straight to the source codes at Bitbucket. Read the artifact’s javadoc at javadoc.io.
Add serial TTY (COM) port support
The refcodes-serial-alt-tty
artifact bridges with jSerialComm to directly interact with your serial ports (aka COM
ports):
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-serial-alt-tty</artifactId>
<groupId>org.refcodes</groupId>
<version>3.3.9</version>
</dependency>
...
</dependencies>
The artifact is hosted directly at Maven Central. Jump straight to the source codes at Bitbucket. Read the artifact’s javadoc at javadoc.io.
Manage request/response handshake with error correction
Last but not least, the refcodes-serial-ext-handshake
artifact provides handshake means in terms of request/response ping-pong alongside with retry and acknowledge management:
1
2
3
4
5
6
7
8
9
<dependencies>
...
<dependency>
<artifactId>refcodes-serial-ext-handshake</artifactId>
<groupId>org.refcodes</groupId>
<version>3.3.9</version>
</dependency>
...
</dependencies>
The artifact is hosted directly at Maven Central. Jump straight to the source codes at Bitbucket. Read the artifact’s javadoc at javadoc.io.
How do I get started?
Use the static import syntactic sugar to easily harness the refcoces-serial
features.
1
2
import static org.refcodes.serial.SerialSugar.*;
// ...
Key players for building data structures fit for serial communication are the types
Segment
as well as Sequence
.
A Segment
is a high level representation for primitive types or complex types, taking care
of endianess, length allocations for
strings or arrays or CRC
checksum validations and is used to construct arbitrary complex data structures.
A Sequence
is the low level representation of a Segment
,
a Segment
can be
converted to a Sequence
forth and back. A Sequence
represents the bytes representation of a Segment
,
with endianess, length allocations or
CRC
checksums already
taken care of as specified by the Segment
and it provides some useful methods to manipulate its bytes representation.
Snippets of interest
Below find some code snippets which demonstrate the various aspects of using the refcodes-serial
artifact (and , if applicable, its offsprings). See also the example source codes of this artifact for further information on the usage of this artifact.
Our first data structure
A Segment
represents a high level data structure capable to be send through a wire (after
being converted to a Sequence
),
containing everything required to pump its bytes representation through a serial
port. This means a Segment
takes care of the endianess, length allocations
for strings or arrays or CRC
checksum validations of your data structure. Constructing such a data structure is
as easy as this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// ...
BooleanSegment theFlag;
IntSegment theValue;
StringSection theString;
Segment theSegment = crcPrefixSegment(
segmentComposite(
theFlag = booleanSegment( true ),
theValue = intSegment( 5161, Endianess.LITTLE_ENDIAN),
allocSegment(
theString = stringSection("Hello world!"), 2, Endianess.LITTLE_ENDIAN
)
), CrcAlgorithmConfig.CRC_16_CCITT_FALSE
);
// ...
The above data structure consists of a boolean value (line 8), an integer value
(line 9) as well as a string value (line 11). All values consisting of more than
a single byte will be encoded using the little endianess
representation (lines 9 and 11). For the length allocation of the string we use two bytes (line 11).
The data structure is secured against data decay during transmission using a CRC-16/CCITT-FALSE
checksum (line 13). Now we can play a little with this data structure:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// ...
// Create the bytes representation:
byte[] theBytes = theSegment.toSequence().toBytes();
System.out.println( NumericalUtility.toHexString( " ", theBytes ) );
System.out.println( theFlag.getPayload() );
System.out.println( theValue.getPayload() );
System.out.println( theString.getPayload() );
// Change the payload and create the bytes representation again:
theFlag.setPayload( false );
byte[] theOtherBytes = theSegment.toSequence().toBytes();
System.out.println( NumericalUtility.toHexString( " ", theOtherBytes ) );
System.out.println( theFlag.getPayload() );
System.out.println( theValue.getPayload() );
System.out.println( theString.getPayload() );
// Restore the data structure from original bytes representation:
theSegment.fromTransmission( theBytes );
System.out.println( theFlag.getPayload() );
System.out.println( theValue.getPayload() );
System.out.println( theString.getPayload() );
// Print our data structure's schema:
System.out.println( theSegment.toSchema() );
// ...
In Line 3 we retrieve the Sequence
representation of the Segment
from which we get it’s bytes representation. The Sequence
we retrieved has already been encoded with the endianess,
the length allocations for strings or arrays as well as the calculated CRC
checksum as specified by the Segment
.
Line 4 prints the final bytes representation of our original data structure:
1
2
0x02 0x58 0x01 0x29 0x14 0x00 0x00 0x0c 0x00 0x48 0x65 0x6c 0x6c 0x6f 0x20 0x77
0x6f 0x72 0x6c 0x64 0x21
In line 10 we change a single boolean value of our payload and line 12 prints out this other bytes representation of our data structure:
1
2
0x1d 0x86 0x00 0x29 0x14 0x00 0x00 0x0c 0x00 0x48 0x65 0x6c 0x6c 0x6f 0x20 0x77
0x6f 0x72 0x6c 0x64 0x21
Comparing the two above hexadecimal representations of our data structure, we see
that the first two bytes as well the third byte differ: The third byte is our boolean
value, represented as a byte (true
= 0x01
, false
= 0x00
) and the first
two bytes are the CRC
checksum: As we have changed our boolean value from true
to false
we got a
completely different checksum, this way we will detect erroneous transmissions (in
most cases).
In line 18 we restore our data structure from the original byte array, having our
boolean value set to true
again.
Finally in line 24 we print out the SerialSchema
of our data structure, useful for debugging and documentation purposes. Here we see the exact construction of our data structure:
The
SerialSchema
is aJSON
alike representation of your expression and helps to further process, analyze, debug or document your expressions.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
CrcSegmentDecorator: {
CRC_ALGORITHM: "CRC-16/CCITT-FALSE",
CRC_BYTE_WIDTH: 2,
CRC_CHECKSUM: 22530,
CRC_CHECKSUM_CONCATENATION_MODE: "PREPEND",
CRC_CHECKSUM_HEX: { 0x02, 0x58 },
CRC_CHECKSUM_LITTLE_ENDIAN_BYTES: { 0x02, 0x58 },
CRC_ENDIANESS: "LITTLE",
DESCRIPTION: "A segment decorator enriching the encapsulated segment with a CRC checksum.",
LENGTH: 21,
TYPE: "org.refcodes.serial.CrcSegmentDecorator",
VALUE: { 0x02, 0x58, 0x01, 0x29, 0x14, 0x00, 0x00, 0x0c, 0x00, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64, 0x21 },
SegmentComposite: {
DESCRIPTION: "A body containing a composite segment as payload.",
LENGTH: 19,
TYPE: "org.refcodes.serial.SegmentComposite",
VALUE: { 0x01, 0x29, 0x14, 0x00, 0x00, 0x0c, 0x00, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64, 0x21 },
BooleanSegment: {
ALIAS: "booleanSegment",
DESCRIPTION: "A segment containing an boolean payload.",
LENGTH: 1,
TYPE: "org.refcodes.serial.BooleanSegment",
VALUE: { 0x01 },
VERBOSE: "true"
},
IntSegment: {
ALIAS: "intSegment",
DESCRIPTION: "A body containing an integer payload.",
ENDIANESS: "LITTLE",
LENGTH: 4,
TYPE: "org.refcodes.serial.IntSegment",
VALUE: { 0x29, 0x14, 0x00, 0x00 },
VERBOSE: "5161"
},
AllocSectionDecoratorSegment: {
ALLOC_LENGTH: 12,
ALLOC_LENGTH_WIDTH: 2,
DESCRIPTION: "An allocation decorator referencing a decoratee and prefixing the length of the decoratee in bytes.",
ENDIANESS: "LITTLE",
LENGTH: 14,
TYPE: "org.refcodes.serial.AllocSectionDecoratorSegment",
VALUE: { 0x0c, 0x00, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64, 0x21 },
StringSection: {
ALIAS: "stringSection",
DESCRIPTION: "A section containing a string payload.",
LENGTH: 12,
TYPE: "org.refcodes.serial.StringSection",
VALUE: { 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x77, 0x6f, 0x72, 0x6c, 0x64, 0x21 },
VERBOSE: "Hello world!"
}
}
}
}
The SerialSchema
gives you full information on the construction of your Segment
.
The above
SerialSchema
tells us that our data structure uses aCRC
checksum (line 1) for detecting transmission errors, calculated with theCRC-16/CCITT-FALSE
algorithm (line 2), the length of the checksum is 2 bytes (line 3) and the checksum calculated for the data structure is22530
(line 4). Moving further down in theJSON
we see the nestedSerialSchema
structures such as the the allocated length for our string (line 36) or the number of bytes used to store the allocation width (line 37) of the string.
Using transmission metrics
To ease our life, we can define a TransmissionMetrics
object declaring the bytes representation of our data structure without the need
to repeat ourself: In the previous example, we have set the CRC
checksum algorithm,
the endianess and the number of bytes
for representing the length of a string or an array (length allocation) individually
for each data type in the data structure respectively. Using a TransmissionMetrics
object, the previous example would now look as follows:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// ...
TransmissionMetrics theMetrics = TransmissionMetrics.builder().withEndianess( Endianess.LITTLE_ENDIAN ).withLengthWidth( 2 ).withCrcAlgorithm( CrcAlgorithmConfig.CRC_16_CCITT_FALSE ).build();
BooleanSegment theFlag;
IntSegment theValue;
StringSection theString;
Segment theSegment = crcPrefixSegment(
segmentComposite(
theFlag = booleanSegment( true ),
theValue = intSegment( 5161, theMetrics ),
allocSegment(
theString = stringSection("Hello world!"), theMetrics
)
), theMetrics
);
// ...
In line 2 we globally defined our CRC
checksum algorithm, the endianess and
the number of bytes for representing the length of a string (length allocation).
This makes live much easier e.g. changing the overall endianess from little to big
is just an issue of changing the TransmissionMetrics
in line 2.
Using a POJO as data structure
Given a Sensor
class which provides data for a single sensor:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
public class Sensor {
public int _value;
public String _name;
public Sensor() {}
public Sensor( String aName, int aValue ) {
_name = aName;
_value = aValue;
}
public String getName() {
return _name;
}
public int getPayload() {
return _value;
}
}
We directly can use this type or even an array of this type to create a bytes representation which we can transmit forth and back (below we actually go for the array):
1
2
3
4
5
// ...
TransmissionMetrics theMetrics = TransmissionMetrics.builder().withEndianess( Endianess.LITTLE_ENDIAN ).withLengthWidth( 2 ).withCrcAlgorithm( CrcAlgorithmConfig.CRC_16_CCITT_FALSE ).build();
Sensor[] theSensors = new Sensor[] { new Sensor( "SensorA", 103343 ), new Sensor( "SensorB", 22109 ), new Sensor( "SensorC", 313773 ) };
ComplexTypeSegment<Sensor[]> theSegment = complexTypeSegment( theSensors, theMetrics );
// ...
Once again in line 2 we globally defined our CRC
checksum algorithm, the endianess and
the number of bytes for representing the length of a string (length allocation).
In line 3 we define an array of Sensor
objects which we want to use for serial
transmission. Finally a Segment
is constructed from the Sensor
objects array in line 4.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// ...
byte[] theBytes = theSegment.toSequence().toBytes();
System.out.println( NumericalUtility.toHexString( " ", theBytes ) );
// Restore the byte representation back to the data structure:
ComplexTypeSegment<Sensor[]> theOtherSegment = complexTypeSegment( Sensor[].class, theMetrics );
theOtherSegment.fromTransmission( theBytes );
Sensor[] theOtherSensors = theOtherSegment.getPayload();
System.out.println( theOtherSensors[0].getName() + "=" + theOtherSensors[0].getPayload() );
System.out.println( theOtherSensors[1].getName() + "=" + theOtherSensors[1].getPayload() );
System.out.println( theOtherSensors[2].getName() + "=" + theOtherSensors[2].getPayload() );
// Print our data structure's schema:
System.out.println( theSegment.toSchema() );
// ...
Having constructed the Segment
in the previous code snippet, we now retrieve the bytes representation of this Sensor
objects array from that very Segment
in line 2. An empty Segment
for restoring the given bytes back to an identical Sensor
objects array is instantiated
in line 6. Eventually the original bytes are fed into this other Segment
and an identical Sensor
objects array is retrieved in line 8.
This example also works perfectly fine with the
Record
type first seen in Java 16.
Under the hood
As you might already have noticed, a unified representation of the serialized bytes
of our data structures, being the Sequence
type, is used to efficiently process and manipulate the byte representations of
our data structures.
Examples
See the funcodes-playload
P2P
(Peer-to-Peer) command line application which makes use of all of the refcodes-serial
, refcodes-serial-alt-tty
as well as refcodes-serial-ext-handshake
artifacts - please also refer to the PLAYLOAD manpage or the downloads section of this site.
For usage information, please take a look at the according examples here.
For usage information on the alternate serial port (aka COM
port) bridging implementation (using jSerialComm), please take a look at the according examples here.
For usage information on the full duplex (request/response) handshake extensions, please take a look at the according examples here.
Contribution guidelines
- Report issues
- Finding bugs
- Helping fixing bugs
- Making code and documentation better
- Enhance the code
Who do I talk to?
- Siegfried Steiner (steiner@refcodes.org)
Terms and conditions
The REFCODES.ORG
group of artifacts is published under some open source licenses; covered by the refcodes-licensing
(org.refcodes
group) artifact - evident in each artifact in question as of the pom.xml
dependency included in such artifact.