Sketch framework and Class Library - Part 1

• Download demo - 3.7 KB

Introduction

This article in three parts looks at the development of a standard framework for a Sketch running on an Arduino board, and a .NET Class Library to encapsulate the interface to the board for a control programme in Windows.

The sketch framework provides a standard way of receiving commands from the control application and of reporting both status information (more or less static) and dynamic values (for example read from input pins on the board).

The Class Library allows the control application developer to be less concerned about the nuts and bolts of driving the serial comms, and to write an application that will work with devices having different capabilities.

Background

In my work (at a University Psychology School) we frequently require to measure various environmental and biometric parameters during research experiments as well as using a range of ‘unusual’ input and output devices to present a stimulus to a participant and detect a response.

This has previously involved a mixture of custom electronic design and interface to a computer controlling the experiment and/or hacking standard devices like keyboards and joysticks to be able to use special buttons or analogue inputs.

We are also typically concerned with reaction times, so minimising lag or jitter affecting timing accuracy can be critical.

When the Arduino was released it was immediately obvious that this could provide a standardised interface to a wide variety of sensors and be able to control all sorts of different output devices.

Various biometric and environmental sensors became available to connect directly to the Arduino, and their values could be presented to the experiment either over the USB serial port, or by configuring the Arduino to emulate a standard keyboard or game controller.

A number of issues quickly became apparent – the most basic being how to consistently identify what sketch and which version was running on any given board. In addition it was useful to be able to ask the firmware what capabilities it had and have the answer returned in a standard format.

Some commands are common across a wide range of applications – for example setting the length of the main sketch program loop, and this led to developing a standard command format so that all experiment applications were able to communicate with the boards in a common way.

This further allowed the development of a class library to encapsulate the communication with the board and remove some of the complications of having to deal with the nuts and bolts of serial communication from the programmer/researcher developing the main experiment software.

This article presents a particular solution to these issues which should be applicable in many different situations beyond our Psychology research programs. If you are using Arduinos with different sensors and controlling different hardware, then having a standard framework can make life a lot simpler.

There are three parts to this. In the first we will define a common command set and protocol and implement a standard sketch framework. At this stage we are not concerned with the control computer end of things, a simple terminal programme is all that is required – although the Arduino IDE own terminal is not capable of sending control characters to the board so you will need something like TeraTerm.

The second part develops a basic class library to encapsulate the interface to any board using the standard framework. A demo control programme is developed in the third part.

Using the code

One of the first decisions was to standardise on a common baud rate for serial communication between the PC and the Arduino. At 9600 baud the bit time on the wire is about 104 microseconds (us) so a single byte (or character) of 10 bits (including start+stop) takes 1.04 milliseconds (ms) to transmit. Since we are interested in timing accuracy down in the millisecond range and sample intervals in the 10’s of milliseconds this was a little slow. We decided to standardise on 115200 baud and to as far as possible use single characters for commands and responses. At this rate a character takes about 87us to transmit on the wire – there is then the low level processing overhead at both ends to drive the serial port and present the received value to the application.

Returning as a string value from a pin will typically use 5 chars which will take nearly 0.5 ms to transmit, In practice that means we will not be able to poll faster than about 2ms loop delay with this protocol, but that is ok for almost all our application.

Having agreed a common baud rate the next issue was simply to be able to identify what firmware was running on a given board.

In order to minimise response times when issuing cmmands we decided that a command should consist of a single character, optionally followed by characters providing any parameters needed, followed by a linefeed. We decided to reserve ASCII control characters (values less than 32) for standard commands, and allow application to use all other characters for their specific needs.

In the standard framework we would store the name of the application, its version number, the date it was compiled, and the developer name. We also decided to automatically give each board a unique name at compile time.

At last some sketch code:

#include <avr/pgmspace.h> 	//enable use of flash memory

static String unitID = "Demo-" + String(__TIME__); 
// __TIME__ has format hh:mm:ss – if we happen to compile at exactly the same time another day we will get the same ID, but pretty unlikely.
						
/* Sketch info strings */
const char sketchName[] PROGMEM = "SoP Demo";	// any valid string
const char sketchVer[] PROGMEM = "1.0.0";		// A.B[.C[.D]] 
const char sketchDate[] PROGMEM = __TIMESTAMP__;		
// __TIMESTAMP__ has format ddd dd MMM hh:mm:ss YYYY - a nightmare to parse!!!
const char sketchAuthor[] PROGMEM = "RogerCO";	// any valid string
const char* const sketchArr[] PROGMEM = { sketchName, sketchVer, sketchDate, sketchAuthor };

char strbuffer[50];  // variable to copy strings from PROGMEN when required

What we have done here is define some standard information as string constants and are saving them in program (flash) memory PROGMEM to save SRAM space.

The UnitID we have stored in SRAM as a variable so that we can, if we wish, change it later.

We are going to use Ctrl S (ASCII 19) as a single character command to which the unit will respond by returning the sketch info.

The main program loop will generally do whatever polling is required on input pins and then check the serial port for any incoming commands and respond as appropriate.

static long loopTime = 20;	// default delay in mS used in main loop

void loop()
{
	//do any read/write ports stuff 

	// deal with any serial commands
	char serialChar;			
	while (Serial.available() > 0) {  
		serialChar = Serial.read();
		if (serialChar = 19) {	// standard commands are Ctrl chars
			reportSketch();
		} else { 	// deal with any sketch specific commands	

	} // endwhile serial port has bytes available

	// do any final tidying up required at end of loop and wait until we go again
	delay(loopTime);

} // end main loop

When we return status values from the sketch we are going to use a standard format of a single character identifier followed by an equals sign followed by the value and a line termination. This will allow us to parse the responses in a standard way.

void reportSketch(void) {
/* reports sketch details */
	strcpy_P(strbuffer, (char*)pgm_read_word(&(sketchArr[0]))); //sketch name
	Serial.print("S="); Serial.println(strbuffer);
	strcpy_P(strbuffer, (char*)pgm_read_word(&(sketchArr[1]))); //sketch version
	Serial.print("V="); Serial.println(strbuffer);
	strcpy_P(strbuffer, (char*)pgm_read_word(&(sketchArr[2]))); //sketch comiled date
	Serial.print("D="); Serial.println(strbuffer);
	strcpy_P(strbuffer, (char*)pgm_read_word(&(sketchArr[3]))); //sketch author
	Serial.print("A="); Serial.println(strbuffer);
	Serial.print("N="); Serial.println(unitID);
}

Here we are reading the information from PROGMEM and writing the identifier and the string to the serial port.

Rather than hard coding the identifiers it would be better to start by defining them all as constants. In general there are two types of data that a sketch will be returning - status information which rarely changes, and dynamic values which are returned every time around the main loop or in response to an interrupt input on the Arduino.

For status information we will identify what is being returned by a single character followed by an equals sign followed by the data followed by a line termination.  As well as sketch name, version, date, author and unit ID shown above there are various other standard commands and responses that we might need. We will be reserving a bunch of upper case identifier characters for standard responses. We will not use control characters for the response labels so that they remain human readable.

If we are polling at more than ten times a second (loop time less than 100ms) and sending values back over the serial port every time around the loop there can be quite a flood of data arriving at the computer. It might be convenient to turn this off or on from the control end. Also if there are pins that are configured as interrupts on the Arduino, or we are using timer interrupts which are generating output on the serial port we might also wish to enable or disable these remotely.

A common requirement turned out to be to adjust the length of the main program loop, and also to report back any errors or jitter in the loop time.

/* variables associated with standard status info */
static unsigned long loopTime = 20;  // default delay in mS used in main loop
static boolean sendSerialValues = false; // default not send values to serial port
static boolean sendSerialInts = false; // default not send interrupts to serial port

We also identified that we would like the sketch to be able to tell us how its input and output pins were being used, what commands it responds to (in addition to the standard set), and what status or values it is returning.

Here is a set of identifiers for the standard control character commands we are using

/* standard ctrl commands */
static const byte cmdLoopOn = 1;	//ctrlA enables loop values out to serial
static const byte cmdIntOn = 2;		//ctrlB enables interrupt values to serial
static const byte qCmdDefn = 3;		//ctrlC report sketch command definitions
static const byte qStatusDefn = 4;	//ctrlD report sketch status definitions
static const byte qValDefn = 5;		//ctrlE report sketch value definitions
static const byte qIns = 6;			//ctrlF report input pins used
static const byte qOuts = 7;		//ctrlG report output pins used
static const byte qSketch = 19;		//ctrlS report sketch name, ver, date, author
static const byte cmdLoopTime = 20;	//ctrlT set or get main loop time in ms
static const byte cmdUnitID = 21;	//ctrlU set or get UnitID
static const byte cmdLoopOff = 24;	//ctrlX disable loop values out to serial
static const byte cmdIntOff = 25;	//ctrlY disable interrupt values to serial
static const byte qErrors = 26;		//ctrlZ report timing errors

and here is a set of corresponding status labels

/* standard status flags */
static const String lblAuthor = "A=";	// Sketch author
static const String lblCmd = "C=";		// Command definition
static const String lblDate = "D=";	    // Sketch Date
static const String lblStatus = "E=";	// Status definition
static const String lblValue = "F=";	// Value definition"
static const String lblIn = "G=";		// Input pin definition"
static const String lblOut = "H=";		// Output pin definition"
static const String lblIntOn = "I=";	// Interrupt output enabled 0|1
static const String lblUnitName = "N=";	// Unit ID
static const String lblLoopOn = "O=";	// Loop output enabled 0|1
static const String lblSketch = "S=";	// Sketch name
static const String lblLoopTime = "T=";	// Main Loop time ms 
static const String lblVersion = "V=";	// Sketch version (as version string format)
static const String lblErrors = "Z=";	// Timing errors report

In addition there will be some sketch specific commands and responses to be defined. For example suppose our board is reading an analogue input and reporting the value back to the control program every time around the loop. It may also be flashing an LED to indicate that a threshold value has been reached. We might want commands to enable or disable the LED output and to set the minimum duration of the LED pulse.

/* example sketch specific commands and descriptors */
static const byte cmdDoLED = 'L';
const char Lcstr[] PROGMEM = "L=Enable LED";
static const byte cmdNoLED = 'l';
const char lcstr[] PROGMEM = "l=Disable LED";
static const byte cmdPulseWidth = 'p';
const char pcstr[] PROGMEM = "p=Set/Get LED pulse duration (ms)";
const char* const cmdArr[] PROGMEM = { Lcstr, lcstr, pcstr };
static int cmdSize = 3;

Here we have defined three commands that this sketch is going to respond to, and their descriptions. Where we are turning a function on or off we will use the upper case command to turn it on and the lower case to turn it off. If a command requires some parameter, eg a value for the pulse width, then it will follow immediately after the command character and be terminated with a line feed.

We will also need to define corresponding status labels and descriptions for the sketch specific values and status info. Again we will store these in PROGMEM.

The definition responses all have a standard format. Each definition is on one line starting with the appropriate label, then the definition then a linefeed. So when the sketch receives the command, eg qCmdDefn it responds by iterating through the list of valid commands for this sketch and returning each one on a separate line preceded by lblCmd.

In response to the command CtrlS as previously seen we respond with the information about the sketch. The response to CrtlC is to return a list of additional commands that the sketch will recognise together with their descriptions. Each command and description will be listed on a separate line starting with "C="

int x;

void reportCommands(void) {
/* returns a list of all of the command characters recognised by this sketch */
	for (x = 0; x < cmdSize; x++) {
		strcpy_P(strbuffer, (char*)pgm_read_word(&(cmdArr[x])));
		Serial.print(lblCmd);
		Serial.println(strbuffer);
	}
}

This function will iterate through the array of command definitions in PROGMEM and output them to the serial port. The control programme will receive this:

Now the control programme can know that if we send to the Arduino an “L” at the beginning of a line it will enable the LED output, and if we send “p500” it will set the pulse width to 500ms.

Similar functions are defined for each of the other standard definition commands CtrlE, CtrlF, CtrlG, and CtrlH: reportStatusCodes(), reportValueCodes(), reportInputPins(), reportOutputPins()

We will return any dynamic values that are reported every time around the loop again using a single character identifier, then a colon ":", the the value as a string of chars terminated with a linefeed. If required we can reduce the output by only returning values when they change, or by holding them in local storage and reading them out in response to a specific command.

/* Sketch value variables, identifier labels and descriptors */
static int anaVal;
static const String lblAnaVal = "A:";
const char avstr[] PROGMEM = "A:,Analogue input value (arbitary units)";
const char* const valArr[] PROGMEM = { avstr };
const int valSize = 1;

If sendSerialValues is true then we will be getting a stream of values from the board

A:126
A:132
A:129
...

Now we can write a generic function to handle control commands and specific functions for each command.

In main programme loop

// ...
while (Serial.available() > 0) {  
		serialChar = Serial.read();
		String cmdstr;
		if (serialByte < 28) {	// standard commands are Ctrl chars
			doStdCmd(serialChar);
		} else { 	// deal with any sketch specific commands	

	} // endwhile serial port has bytes available
// ...

 

The doStdCmd() function to handle the standard commands (ascii values less than 32) and an example function reading a parameter value after the command to set the loop time

void doStdCmd(char cmd) {
/* handles a standard control command */
	switch (cmd) {
    case qSketch:	
        reportSketch();
		break;
	case cmdLoopOn:
		sendSerialValues = true;
		Serial.print(lblLoopOn); Serial.println(sendSerialValues);
		break;
// other command function calls as required - see full example
	case cmdLoopTime:	// in this case we are expecting a parameter value
		setLoopTime();
// we always report back the current loop time, so sending just CtrlT will give us the current value without changing it
		Serial.print(lblLoopTime); Serial.println(loopTime);
		break;
	}
}

void setLoopTime (void) {
/* example of handling a command that has a numeric parameter */
	boolean endOfDigits = false;
	String cmdStr = "";
int inChar;
	while (!endOfDigits && Serial.available()) { 
//read digits from the port while available until a non-digit char found
		inChar = Serial.read();
		if (isDigit(inChar)) {
			cmdStr += (char)inChar;
		} else {
			endOfDigits = true;
		}
	}
	if (cmdStr.length() >= 1) { // if we have a number > 0 set new loopTime
		long tmp = cmdStr.toInt();
		if (tmp > 0) loopTime = tmp; 
	}
}

OK, assuming we have defined status, value, input and output definitions as required in PROGMEM and written appropriate functions reportStatusCodes() etc. then we should have a basic framework and an idea of how to handle sketch specific commands. These are shown in the full sample framework code to download.

One final thing, we happen to be a bit concerned to ensure that the main programme loop always takes the same amount of time to execute whatever complex command processing or interrupt handling or value conversion might have been involved during a particular trip round the loop.

In simple terms we will note the time from the built in millisecond timer, millis(), at the start and end of the loop and adjust the delay we are using accordingly.

/* variables to adjust length of timing loop */
unsigned long loopStart;
unsigned long loopEnd;
static unsigned long loopTime = 20;	// default delay in mS used in main loop
int exeTime;

void loop()
{
	loopStart = millis();

	// do other stuff as required

	// correct timing errors
	loopEnd = millis();
	exeTime = loopEnd - loopStart;
//trap error if millis() wraps around during this loop, this will happen every few days, you can calculate how often if you read the documentation ...
	if (exeTime > 0) { exeTime = 0; }  
	if (exeTime > loopTime) 
		delay(loopTime - exeTime);
	} else {			
// if the actual loop execution time is greater than the desired delay then we have overrun so no delay and optionally write some code here to track the errors and report them in response to a qErrors command ... 
	}

} // end main loop

That’s it. We’ve now got a framework for any sketch that will respond to some standard commands and enable us to identify what is running and what its capabilities are.

A final word on tools. Obviously you can write sketch code in any development environment you like, including the Arduino native IDE, which you also need to compile and upload the code to the board.

Since the bulk of the rest of our work is in Windows we make extensive use of Visual Studio and I would recommend the Visual Micro addon to Visual Studio. It encapsulates the Ardunio development tools inside Visual Studio and gives you access to Intellisense and all the other tools you are used to. You do have to have C++ installed in Visual Studio.

In part 2 we will use facilities provided by this framework to build a class library that encapsulates interfacing to the Arduino for a Windows Forms Application, including identifying which virtual serial port the Arduino is connected to and presenting its status information and values in a consistent manner.

Points of Interest

We have defined a standard communication protocol for commands to the Arduino board and returning both status information and dynamic values.

We have used PROGMEM to store string constants out of the way. In a future article we will look at using the EEPROM to enable us to preserve status variables when the power is cycled on the board.

We can interogate the Sketch to ask what pins it is using, what commands it responds to, what status information and values it returns.

We have tried to ensure that the execution time for the main programme loop is consistent whatever processing happens during each trip around the loop. So if we are polling a sensor every 20ms we will get a result returned from the Arduino every 20ms - of course the overhead in the operating system of the control computer may be variable, that is a whole different topic. Neither Windows nor Linux/MacOS are real-time operating systems for the application developer.

If you are producing multiple different Arduino projects with different functionality then using a standard framework for all your sketches can make life a lot easier in the long run. Of course there is an overhead in terms of needing to include the standard code every time, but if you are a competent C programmer you will find how to put all of the standard functions into a library that you can simply #include at the top of each project.

History

First version submitted 26th July 2016, typos corrected and minor adds 27th July

Updated code 1st August to tidy some errors and add comments