INTRODUCTION
I am trying to make an ESP8266 based Arduino Serial Uploader using its SPI File System.
(Project ongoing || I will post the project link as soon as I finish it. But, you can always follow the GitHub repo)
Since I plan to upload the Arduino IDE .hex
compile output directly to the file system, I need to parse it and give byte*
output before uploading it to the system flash.
You must be wondering why hex and not binary straightaway?
Well, if you want to do some editing on the output file, .bin
displays garbage in the text editor.
HEX files (basically ASCII text files) are larger and more complicated than binary files because they consist of the binary data plus various information like memory addresses, where the data needs to be written.
Example File:
There are many HEX parser libraries available out there, but I needed a bare minimum one.
So, let’s get started.
EXPLANATION
You can go through the nitty-gritty of the Intel HEX format here and understand the nomenclature/characteristics.
After you are able to read a HEX file, the C++ implementation should be a walk in the park.
I made a HEXparser
class to use it later in my Web Server code.
Here’s the class declaration:
class HEXparser { public: HEXparser(); void ParseRecord(byte* data); // Import data from spiffs.open() in web server. byte* FetchRaw(); // will be called from main server code byte* FetchAddress(); // for flashing private: int recType = 0; // Here only 0/1 is needed-data present/end of file. int length = 0; int index = 0; // to keep track of the no. of bytes in each chunk byte RawPage[128]; // sending 128 bytes of 'chunk' each time byte address[2]; // storage array bool firstTime = true; // data address is to be updated only after the first chunk transfer /* Intel HEX file format info */ int RecordLength(byte* record); void RecordAddress(byte* record); int RecordType(byte* record); // // void extractData(byte* record, int len); void FileEnd(); };
THE GITHUB REPOSITORY
CODE BREAKDOWN
We basically loop over a given file, going through each line/HEX record and parsing them.
1. How each record will be extracted from the file (I have not written the server code yet. This is just for reference):
void Server::proto(WiFiClient* client, String filen) { SPIFFS.begin(); File file = SPIFFS.open(filen, "r"); if(file) { HEXparser hexParse; while(file.available()) { byte holder[50]; String data = file.readStringUntil('\n'); data.getBytes(holder, data.length()); hexParse.ParseRecord(holder); if(hexParse.IsPageReady()) { byte* page = hexParse.FetchRaw(); byte* address = hexParse.FetchAddress(); } } } file.close(); SPIFFS.end(); }
2. Now, let’s parse the record.
This is the main function. All the function calls are explained after this.
void HEXparser::ParseRecord(byte* record) { recType = RecordType(record); if(recType == 0) // data present { length = RecordLength(record); extractData(record, length); if(index == 128) // chunk full { if(!firstTime) { address[1] += 0x40; //update data address if(address[1] == 0) { address[0] += 0x1; } } index = 0; firstTime = false; } } if(recType == 1) // end of file { FileEnd(); } }
3. Analyzing the record
Extracting useful information about the record, following the Intel format:
// length = number of data bytes int HEXparser::RecordLength(byte* record) { char holder[3]; holder[0] = record[1]; holder[1] = record[2]; holder[2] = '\0'; return strtol(holder, 0, 16); } // the address where the corresponding record data are to be allocated in memory. void HEXparser::RecordAddress(byte* record) { char holder[3]; holder[2] = '\0'; holder[0] = record[3]; holder[1] = record[4]; address[0] = strtol(buff, 0, 16); holder[0] = record[5]; holder[1] = record[6]; address[1] = strtol(buff, 0, 16); } // data or end of file int HEXparser::RecordType(byte* record) { char holder[3]; holder[0] = record[7]; holder[1] = record[8]; holder[2] = '\0'; return strtol(holder, 0, 16); }
4. Extracting and preparing the program data to be flash-ready.
First, we need to convert the whole data to bytes.
For this, a flash page is created, which holds the data bytes. Now, data is not sent continuously through the STK500v1 protocol (which I’ll be using to program).
It needs to be chunks of data, 128 bytes of them ( source: avrdude.conf
from arduino IDE files).
// each record is passed void HEXparser::extractData(byte* record, int len) { int begin = 9; // start position of data in record int end = (len*2) + begin; //data is in sets of 2 char holder[3]; holder[2] = '\0'; for(int i=begin;begin<end;i=i+2) { holder[0] = record[i]; holder[1] = record[i+1]; RawPage[index] = strtol(holder, 0, 16); index++; // keeping track of chunk size } }
Configuring Chunks
- Addresses are updated once index exceeds 127 to form a new page/chunk.
-
I came across another case where we need to control the page index. When we are done traversing the whole file, but index is less than 127, we need to fill the unused areas with the specified constant value. A fill value of 0xFF is often used with this option because it corresponds to erased flash.
// check record type||chunk filled or not void HEXparser::FileEnd() { address[1] += 0x40; if(address[1] == 0) { address[0] += 0x1; } while(index<128) { RawPage[index] = 0xFF; index++ ; } }
5. Finally, sending the RawPage
and address
to the main server code, used while flashing the page.
/* Return calls to spiff/web server */ byte* HEXparser::FetchRaw() { return RawPage; } byte* HEXparser::FetchAddress() { return address; }
Sir,
Please help me understand SPIFFS.
I’m a beginner with ESP and I have done nothing other than flashing it with simple sketches using Arduino Core (using the Arduino IDE).
So I request you to send me some relevant links for understanding SPIFFS. Or write a tutorial on it, sir.
Sir Please.
Thank You.
-Roberto Ricci
LikeLike
Have a look at this instructable: http://www.instructables.com/id/Using-ESP8266-SPIFFS/ .
This should suffice.
LikeLike