![]() |
px-fwlib 0.10.0
Cross-platform embedded library and documentation for 8/16/32-bit microcontrollers generated with Doxygen 1.9.2
|
Record-based file system for Serial Flash.
File(s):
px_log_fs is a basic but resilient record-based file system to store sequential log data on Serial Flash, for example Adesto AT25S Serial Flash. It is designed with the following fundamental characteristics in mind:
File system features:
A file is defined as a collection of sequential records that contain fixed-size log data. The records can be navigated forwards or backwards to read data, while still being able to append new data. New records will overwrite oldest records when the file is full (PX_LOG_FS_CFG_STOP_WR_WHEN_FULL = 0) or record writing will stop when the file is full (PX_LOG_FS_CFG_STOP_WR_WHEN_FULL = 1).
The size of each record is configured with PX_LOG_FS_CFG_REC_DATA_SIZE. See 'px_log_fs_cfg_template.h'
The erase block size (number of pages) is configurable with PX_LOG_FS_CFG_ERASE_BLOCK_SIZE. For example a single page of the Adesto AT45DB041E can be erased at a time (very convenient), but the AT25SF041 has a minimum erase block size of 4 KB or 8 pages (a much cheaper option).
The file system must start at the beginning of an erase block and end at the end of an erase block. It must be at least two erase blocks in size, but more is better.
First a code example to demonstrate the API, followed by implementation details...
The Serial Flash pages are treated as one big circular buffer. Records are stored sequentially and record writing will stop (or wrap to overwrite oldest records) when the start of the oldest record is reached. This simplified scheme implicitly allows for wear leveling, because each Serial Flash page is erased once per pass through the circular buffer.
The file system leverages a characteristic of Serial Flash. When a block of pages are erased, all the data bits are reset to 1. When writing a byte, bits can be cleared (0), but not reset to 1 again. The following 8-bit marker values are used:
By inspecting the bits, it can be observed that a marker can be changed from FREE to PAGE to PAGE+A to BAD, but not the other way around without erasing the whole block.
Each written page starts with a header (px_log_fs_header_t) which consists of an 8-bit marker, a 16-bit incrementing number and an 8-bit CRC. Only the first 4 bytes of each page is read to quickly index the whole file system during px_log_fs_init(). The incrementing number is used to figure out which is the oldest page and which is the newest during px_log_fs_init().
Pages containing records are marked with PAGE (0x5F). Each record (px_log_fs_record_t) contains an 8-bit RECORD marker (0xAF), the record data (fixed size set with PX_LOG_FS_CFG_REC_DATA_SIZE) and an 8-bit CRC.
A record cannot span across pages, so there may be unused space at the end of each page. For example, if a page is 256 bytes and the page header takes up 4 bytes, then there are 252 bytes left over for records. If 8 data bytes are stored in each record, then the total record size is 10 bytes (including overhead). This means that 25 records can be stored in each page and 2 bytes will be unused (252 - 25*10 = 2).
To figure out which page contains the oldest records (FIRST page) and which page contains the newest records (LAST page), a rolling (or wrapping) number is assigned to each page. This means that each consecutive page is labelled with an incrementing number. If the number exceeds the maximum, it starts again at zero (rolls over).
To simplify these examples, the rolling number starts at 00, increments up to 99 and then rolls over to 00.
Each page's rolling number is compared with the next page's rolling number. The largest difference in the rolling numbers indicates that the LAST page has been found. The next valid page is per implication the FIRST.
To be exact, modulo unsigned integer arithmetic is used:
Diff = (rolling_number_next - rolling_number) mod 100
Here are a few select test cases to demonstrate that it works under various conditions:
Oldest records can be marked as archived (RECORD+A marker = 0xAA) for example when the record has been succesfully uploaded to a server. This makes it possible to find the next (unarchived) record that must be uploaded to the server after a reset or power interruption.
If all the records on a page is marked as archived then the page is marked as archived too (PAGE+A marker = 0x55). This makes it faster to find the first unarchived record, because each page marker is inspected until the first unarchived page (PAGE marker = 0x5F) is found. Then each record on that page is inspected until the first unarchived record is found (RECORD marker =0xAF).
Data Structures | |
struct | px_log_fs_adr_t |
Specification of data address in Serial Flash. More... | |
struct | px_log_fs_handle_t |
Macros | |
#define | PX_LOG_FS_CFG_VER 2 |
Config version. More... | |
#define | PX_LOG_FS_CFG_PAGE_SIZE PX_AT25S_PAGE_SIZE |
Page size for file system. More... | |
#define | PX_LOG_FS_CFG_ERASE_BLOCK_SIZE PX_AT25S_PAGES_PER_BLOCK_4KB |
Erase block size (in pages) for file system. More... | |
#define | PX_LOG_FS_CFG_REC_DATA_SIZE 13 |
Record data size (total record size = PX_LOG_FS_CFG_REC_DATA_SIZE + 3 bytes overhead) More... | |
#define | PX_LOG_FS_CFG_STOP_WR_WHEN_FULL 0 |
Stop writing when full (1) or erase oldest records and continue writing (0) More... | |
Enumerations | |
enum | px_log_fs_err_t { PX_LOG_FS_ERR_NONE = 0 , PX_LOG_FS_ERR_EMPTY , PX_LOG_FS_ERR_FULL , PX_LOG_FS_ERR_WRITE_FAIL , PX_LOG_FS_ERR_NO_RECORD , PX_LOG_FS_ERR_FATAL } |
Error codes. More... | |
Functions | |
px_log_fs_err_t | px_log_fs_init (px_log_fs_handle_t *handle, uint16_t fs_page_start, uint16_t fs_page_end) |
Initialise log file system. More... | |
px_log_fs_err_t | px_log_fs_reset (px_log_fs_handle_t *handle, uint16_t fs_page_start, uint16_t fs_page_end) |
Reset log file system. More... | |
px_log_fs_err_t | px_log_fs_rd_first (px_log_fs_handle_t *handle, void *data, size_t nr_of_bytes) |
Read first (oldest) record. More... | |
px_log_fs_err_t | px_log_fs_rd_first_unarchived (px_log_fs_handle_t *handle, void *data, size_t nr_of_bytes) |
Read first (oldest) unarchived record. More... | |
px_log_fs_err_t | px_log_fs_rd_next (px_log_fs_handle_t *handle, void *data, size_t nr_of_bytes) |
Read next (newer) record. More... | |
px_log_fs_err_t | px_log_fs_rd_last (px_log_fs_handle_t *handle, void *data, size_t nr_of_bytes) |
Read last (newest) record. More... | |
px_log_fs_err_t | px_log_fs_rd_previous (px_log_fs_handle_t *handle, void *data, size_t nr_of_bytes) |
Read previous (older) record. More... | |
bool | px_log_fs_rd_rec_is_archived (px_log_fs_handle_t *handle) |
See if currently read record is archived or not. More... | |
px_log_fs_err_t | px_log_fs_rd_rec_set_archive (px_log_fs_handle_t *handle) |
Set currently read record as archived. More... | |
px_log_fs_err_t | px_log_fs_wr (px_log_fs_handle_t *handle, const void *data, size_t nr_of_bytes) |
Write a record to the file. More... | |
void | px_log_fs_dbg_report_info (px_log_fs_handle_t *handle) |
Report log file system info. More... | |
void | px_log_fs_glue_rd (void *buffer, uint16_t page, uint16_t start_byte_in_page, uint16_t nr_of_bytes) |
Read data from Serial Flash. More... | |
void | px_log_fs_glue_wr (const void *buffer, uint16_t page, uint16_t start_byte_in_page, uint16_t nr_of_bytes) |
Writes data to Serial Flash. More... | |
void | px_log_fs_glue_erase_block (uint16_t page) |
Erases data in Serial Flash. More... | |
struct px_log_fs_adr_t |
Specification of data address in Serial Flash.
Definition at line 246 of file px_log_fs.h.
Data Fields | ||
---|---|---|
uint16_t | page | Page. |
uint16_t | offset | Offset inside page. |
struct px_log_fs_handle_t |
Definition at line 252 of file px_log_fs.h.
Data Fields | ||
---|---|---|
uint16_t | fs_page_start | First page in file system (must be on the start of an erase block) |
uint16_t | fs_page_end | Last page in file system (must be on the end of an erase block) |
uint16_t | page_first | First page with records (PX_LOG_FS_PAGE_INVALID if empty) |
uint16_t | page_last | Last page with records (PX_LOG_FS_PAGE_INVALID if empty) |
uint16_t | page_nr_next | Next page number to use (starts at 0) |
px_log_fs_adr_t | adr_wr | Next write address (open position) |
px_log_fs_adr_t | adr_rd | Current read address. |
bool | archive_flag | Flag is set if current read record is archived. |
#define PX_LOG_FS_CFG_VER 2 |
Config version.
Definition at line 35 of file px_log_fs_cfg_template.h.
#define PX_LOG_FS_CFG_PAGE_SIZE PX_AT25S_PAGE_SIZE |
Page size for file system.
Definition at line 38 of file px_log_fs_cfg_template.h.
#define PX_LOG_FS_CFG_ERASE_BLOCK_SIZE PX_AT25S_PAGES_PER_BLOCK_4KB |
Erase block size (in pages) for file system.
Definition at line 41 of file px_log_fs_cfg_template.h.
#define PX_LOG_FS_CFG_REC_DATA_SIZE 13 |
Record data size (total record size = PX_LOG_FS_CFG_REC_DATA_SIZE + 3 bytes overhead)
Definition at line 44 of file px_log_fs_cfg_template.h.
#define PX_LOG_FS_CFG_STOP_WR_WHEN_FULL 0 |
Stop writing when full (1) or erase oldest records and continue writing (0)
Definition at line 47 of file px_log_fs_cfg_template.h.
enum px_log_fs_err_t |
Error codes.
Definition at line 235 of file px_log_fs.h.
px_log_fs_err_t px_log_fs_init | ( | px_log_fs_handle_t * | handle, |
uint16_t | fs_page_start, | ||
uint16_t | fs_page_end | ||
) |
Initialise log file system.
handle | Pointer to file system handle structure |
fs_page_start | First page in file system (must be on the start of an erase block) |
fs_page_end | Last page in file system (must be on the end of an erase block) |
PX_LOG_FS_ERR_NONE | File system was succesfully indexed |
Definition at line 452 of file px_log_fs.c.
px_log_fs_err_t px_log_fs_reset | ( | px_log_fs_handle_t * | handle, |
uint16_t | fs_page_start, | ||
uint16_t | fs_page_end | ||
) |
Reset log file system.
handle | Pointer to file system handle structure |
fs_page_start | First page in file system (must be on the start of an erase block) |
fs_page_end | Last page in file system (must be on the end of an erase block) |
PX_LOG_FS_ERR_NONE | File system was succesfully resetted |
Definition at line 550 of file px_log_fs.c.
px_log_fs_err_t px_log_fs_rd_first | ( | px_log_fs_handle_t * | handle, |
void * | data, | ||
size_t | nr_of_bytes | ||
) |
Read first (oldest) record.
handle | Pointer to file system handle structure |
data | Pointer to buffer where record data must be copied to |
nr_of_bytes | Number of bytes to copy from record |
PX_LOG_FS_ERR_NONE | Valid record data found and copied into structure |
PX_LOG_FS_ERR_NO_RECORD | No record found |
Definition at line 607 of file px_log_fs.c.
px_log_fs_err_t px_log_fs_rd_first_unarchived | ( | px_log_fs_handle_t * | handle, |
void * | data, | ||
size_t | nr_of_bytes | ||
) |
Read first (oldest) unarchived record.
handle | Pointer to file system handle structure |
data | Pointer to buffer where record data must be copied to |
nr_of_bytes | Number of bytes to copy from record |
PX_LOG_FS_ERR_NONE | Valid record data found and copied into structure |
PX_LOG_FS_ERR_NO_RECORD | No record found |
Definition at line 624 of file px_log_fs.c.
px_log_fs_err_t px_log_fs_rd_next | ( | px_log_fs_handle_t * | handle, |
void * | data, | ||
size_t | nr_of_bytes | ||
) |
Read next (newer) record.
handle | Pointer to file system handle structure |
data | Pointer to buffer where record data must be copied to |
nr_of_bytes | Number of bytes to copy from record |
PX_LOG_FS_ERR_NONE | Valid record data found and copied into structure |
PX_LOG_FS_ERR_NO_RECORD | No record found |
Definition at line 675 of file px_log_fs.c.
px_log_fs_err_t px_log_fs_rd_last | ( | px_log_fs_handle_t * | handle, |
void * | data, | ||
size_t | nr_of_bytes | ||
) |
Read last (newest) record.
handle | Pointer to file system handle structure |
data | Pointer to buffer where record data must be copied to |
nr_of_bytes | Number of bytes to copy from record |
PX_LOG_FS_ERR_NONE | Valid record data found and copied into structure |
PX_LOG_FS_ERR_NO_RECORD | No record found |
Definition at line 739 of file px_log_fs.c.
px_log_fs_err_t px_log_fs_rd_previous | ( | px_log_fs_handle_t * | handle, |
void * | data, | ||
size_t | nr_of_bytes | ||
) |
Read previous (older) record.
handle | Pointer to file system handle structure |
data | Pointer to buffer where record data must be copied to |
nr_of_bytes | Number of bytes to copy from record |
PX_LOG_FS_ERR_NONE | Valid record data found and copied into structure |
PX_LOG_FS_ERR_NO_RECORD | No record found |
Definition at line 755 of file px_log_fs.c.
|
inline |
See if currently read record is archived or not.
NB! Record MUST have been succesfully read first.
handle | Pointer to file system handle structure |
true | record is marked as archived |
false | record is NOT marked as archived |
Definition at line 382 of file px_log_fs.h.
px_log_fs_err_t px_log_fs_rd_rec_set_archive | ( | px_log_fs_handle_t * | handle | ) |
Set currently read record as archived.
NB! Record MUST have been succesfully read first.
As a side effect, if this is the last record in the page, then the page marker is set as archived for quick indexing.
It is important that the records are marked as archived starting from the oldest to the newest in sequence otherwise the algorithm that mark a whole page as archived will not work correctly.
handle | Pointer to file system handle structure |
PX_LOG_FS_ERR_NONE | Success. Record is set as archived. |
PX_LOG_FS_ERR_WRITE_FAIL | Failed to set record as archived. |
Definition at line 818 of file px_log_fs.c.
px_log_fs_err_t px_log_fs_wr | ( | px_log_fs_handle_t * | handle, |
const void * | data, | ||
size_t | nr_of_bytes | ||
) |
Write a record to the file.
A new record is appended to the end of the list of records. The maximum number of bytes that can be stored in the record is specified by PX_LOG_FS_CFG_DATA_SIZE.
handle | Pointer to file system handle structure |
data | Pointer to buffer containing data that must be stored in the record |
nr_of_bytes | Number of bytes that must be written |
PX_LOG_FS_ERR_NONE | Success |
PX_LOG_FS_ERR_FULL | File (or file system) is full |
PX_LOG_FS_ERR_WRITE_FAIL | Failed to write record (verify failed) |
Definition at line 897 of file px_log_fs.c.
void px_log_fs_dbg_report_info | ( | px_log_fs_handle_t * | handle | ) |
Report log file system info.
handle | Pointer to file system handle structure |
Definition at line 1000 of file px_log_fs.c.
void px_log_fs_glue_rd | ( | void * | buffer, |
uint16_t | page, | ||
uint16_t | start_byte_in_page, | ||
uint16_t | nr_of_bytes | ||
) |
Read data from Serial Flash.
This glue function reads data from Serial Flash and stores it in the specified buffer.
buffer | Buffer to store read data |
page | PX_LOG_FS_CFG_PAGE_START to PX_LOG_FS_CFG_PAGE_END |
start_byte_in_page | 0 to PX_LOG_FS_CFG_PAGE_SIZE - 1 |
nr_of_bytes | 1 to PX_LOG_FS_CFG_PAGE_SIZE |
Definition at line 41 of file px_log_fs_glue_at25s.c.
void px_log_fs_glue_wr | ( | const void * | buffer, |
uint16_t | page, | ||
uint16_t | start_byte_in_page, | ||
uint16_t | nr_of_bytes | ||
) |
Writes data to Serial Flash.
This glue function write data from Serial Flash and stores it in the specified buffer.
buffer | Buffer to store read data |
page | PX_LOG_FS_CFG_PAGE_START to PX_LOG_FS_CFG_PAGE_END |
start_byte_in_page | 0 to PX_LOG_FS_CFG_PAGE_SIZE - 1 |
nr_of_bytes | 1 to PX_LOG_FS_CFG_PAGE_SIZE |
Definition at line 54 of file px_log_fs_glue_at25s.c.
void px_log_fs_glue_erase_block | ( | uint16_t | page | ) |
Erases data in Serial Flash.
This glue function erases a block of pages in Serial Flash.
page | PX_LOG_FS_CFG_PAGE_START to PX_LOG_FS_CFG_PAGE_END |
Definition at line 67 of file px_log_fs_glue_at25s.c.