 |
Mbed OS Reference
|
|
Mbed OS Reference
|
The goal of this code is to make buffer and pointer manipulation easier and safer when working with binary data. More...
#include <stdint.h>#include <string.h>#include <stddef.h>Go to the source code of this file.
Data Structures | |
| struct | useful_buf_c |
| UsefulBufC and UsefulBuf are simple data structures to hold a pointer and length for a binary data. More... | |
| struct | useful_buf |
| The non-const UsefulBuf typically used for some allocated memory that is to be filled in. More... | |
| struct | useful_out_buf |
| UsefulOutBuf is a structure and functions (an object) that are good for serializing data into a buffer such as is often done with network protocols or data written to files. More... | |
| struct | useful_input_buf |
Macros | |
| #define | NULLUsefulBufC ((UsefulBufC) {NULL, 0}) |
| A "NULL" UsefulBufC is one that has no value in the same way a NULL pointer has no value. More... | |
| #define | NULLUsefulBuf ((UsefulBuf) {NULL, 0}) |
| A NULL UsefulBuf is one that has no memory associated the say way NULL points to nothing. More... | |
| #define | UsefulBuf_FROM_SZ_LITERAL(szString) ((UsefulBufC) {(szString), sizeof(szString)-1}) |
| Convert a literal string to a UsefulBufC. More... | |
| #define | UsefulBuf_FROM_BYTE_ARRAY_LITERAL(pBytes) ((UsefulBufC) {(pBytes), sizeof(pBytes)}) |
| Convert a literal byte array to a UsefulBufC. More... | |
| #define | UsefulBuf_MAKE_STACK_UB(name, size) |
| Make an automatic variable with name of type UsefulBuf and point it to a stack variable of the give size. More... | |
| #define | UsefulBuf_FROM_BYTE_ARRAY(pBytes) ((UsefulBuf) {(pBytes), sizeof(pBytes)}) |
| Make a byte array in to a UsefulBuf. More... | |
| #define | UsefulOutBuf_MakeOnStack(name, size) |
| Convenience marco to make a UsefulOutBuf on the stack and initialize it with stack buffer. More... | |
| #define | UIB_MAGIC (0xB00F) |
| UsefulInputBuf is the counterpart to UsefulOutBuf and is for parsing data read or received. More... | |
Typedefs | |
| typedef struct useful_buf_c | UsefulBufC |
| UsefulBufC and UsefulBuf are simple data structures to hold a pointer and length for a binary data. More... | |
| typedef struct useful_buf | UsefulBuf |
| The non-const UsefulBuf typically used for some allocated memory that is to be filled in. More... | |
| typedef struct useful_out_buf | UsefulOutBuf |
| UsefulOutBuf is a structure and functions (an object) that are good for serializing data into a buffer such as is often done with network protocols or data written to files. More... | |
Functions | |
| UsefulBufC | UsefulBuf_CopyOffset (UsefulBuf Dest, size_t uOffset, const UsefulBufC Src) |
| Copy one UsefulBuf into another at an offset. More... | |
| int | UsefulBuf_Compare (const UsefulBufC UB1, const UsefulBufC UB2) |
| Compare two UsefulBufCs. More... | |
| size_t | UsefulBuf_FindBytes (UsefulBufC BytesToSearch, UsefulBufC BytesToFind) |
| Find one UsefulBuf in another. More... | |
| void | UsefulOutBuf_Init (UsefulOutBuf *me, UsefulBuf Storage) |
| Initialize and supply the actual output buffer. More... | |
| void | UsefulOutBuf_InsertUsefulBuf (UsefulOutBuf *me, UsefulBufC NewData, size_t uPos) |
| Inserts bytes into the UsefulOutBuf. More... | |
| UsefulBufC | UsefulOutBuf_OutUBuf (UsefulOutBuf *me) |
| Returns the resulting valid data in a UsefulOutBuf. More... | |
| UsefulBufC | UsefulOutBuf_CopyOut (UsefulOutBuf *me, UsefulBuf Dest) |
| Copies the valid data out into a supplied buffer. More... | |
| const void * | UsefulInputBuf_GetBytes (UsefulInputBuf *me, size_t uNum) |
| Get pointer to bytes out of the input buffer. More... | |
The goal of this code is to make buffer and pointer manipulation easier and safer when working with binary data.
You use the UsefulBuf, UsefulOutBuf and UsefulInputBuf structures to represent buffers rather than ad hoc pointers and lengths.
With these it will often be possible to write code that does little or no direct pointer manipulation for copying and formatting data. For example the QCBOR encoder was rewritten using these and has no direct pointer manipulation.
While it is true that object code using these functions will be a little larger and slower than a white-knuckle clever use of pointers might be, but not by that much or enough to have an affect for most use cases. For security-oriented code this is highly worthwhile. Clarity, simplicity, reviewability and are more important.
There are some extra sanity and double checks in this code to help catch coding errors and simple memory corruption. They are helpful, but not a substitute for proper code review, input validation and such.
This code consists of a lot of inline functions and a few that are not. It should not generate very much object code, especially with the optimizer turned up to -Os or -O3. The idea is that the inline functions are easier to review and understand and the optimizer does the work of making the code small.
Definition in file UsefulBuf.h.
| #define NULLUsefulBufC ((UsefulBufC) {NULL, 0}) |
A "NULL" UsefulBufC is one that has no value in the same way a NULL pointer has no value.
A UsefulBuf is NULL when the ptr field is NULL. It doesn't matter what len is. See UsefulBuf_IsEmpty() for the distinction between NULL and empty.
Definition at line 171 of file UsefulBuf.h.
| #define NULLUsefulBuf ((UsefulBuf) {NULL, 0}) |
A NULL UsefulBuf is one that has no memory associated the say way NULL points to nothing.
It does not matter what len is.
Definition at line 176 of file UsefulBuf.h.
| #define UsefulBuf_FROM_SZ_LITERAL | ( | szString | ) | ((UsefulBufC) {(szString), sizeof(szString)-1}) |
Convert a literal string to a UsefulBufC.
szString must be a literal string that you can take sizeof. This is better for literal strings than UsefulBuf_FromSZ() because it generates less code. It will not work on non-literal strings.
The terminating \0 (NULL) is NOT included in the length!
Definition at line 299 of file UsefulBuf.h.
| #define UsefulBuf_FROM_BYTE_ARRAY_LITERAL | ( | pBytes | ) | ((UsefulBufC) {(pBytes), sizeof(pBytes)}) |
Convert a literal byte array to a UsefulBufC.
pBytes must be a literal string that you can take sizeof. It will not work on non-literal arrays.
Definition at line 310 of file UsefulBuf.h.
| #define UsefulBuf_MAKE_STACK_UB | ( | name, | |
| size | |||
| ) |
Make an automatic variable with name of type UsefulBuf and point it to a stack variable of the give size.
Definition at line 318 of file UsefulBuf.h.
| #define UsefulBuf_FROM_BYTE_ARRAY | ( | pBytes | ) | ((UsefulBuf) {(pBytes), sizeof(pBytes)}) |
Make a byte array in to a UsefulBuf.
Definition at line 326 of file UsefulBuf.h.
| #define UsefulOutBuf_MakeOnStack | ( | name, | |
| size | |||
| ) |
Convenience marco to make a UsefulOutBuf on the stack and initialize it with stack buffer.
Definition at line 669 of file UsefulBuf.h.
| #define UIB_MAGIC (0xB00F) |
UsefulInputBuf is the counterpart to UsefulOutBuf and is for parsing data read or received.
Initialize it with the data from the network and its length. Then use the functions here to get the various data types out of it. It maintains a position for getting the next item. This means you don't have to track a pointer as you get each object. UsefulInputBuf does that for you and makes sure it never goes off the end of the buffer. The QCBOR implementation parser makes use of this for all its pointer math and length checking.
UsefulInputBuf also maintains an internal error state so you do not have to. Once data has been requested off the end of the buffer, it goes into an error state. You can keep calling functions to get more data but they will either return 0 or NULL. As long as you don't dereference the NULL, you can wait until all data items have been fetched before checking for the error and this can simplify your code.
The integer and float parsing expects network byte order (big endian). Network byte order is what is used by TCP/IP, CBOR and most internet protocols.
Lots of inlining is used to keep code size down. The code optimizer, particularly with the -Os, also reduces code size a lot. The only non-inline code is UsefulInputBuf_GetBytes() which is less than 100 bytes so use of UsefulInputBuf doesn't add much code for all the messy hard-to-get right issues with parsing in C that is solves.
The parse context size is: 64-bit machine: 16 + 8 + 2 + 1 (5 bytes padding to align) = 32 bytes 32-bit machine: 8 + 4 + 2 + 1 (1 byte padding to align) = 16 bytes
Definition at line 1200 of file UsefulBuf.h.
| typedef struct useful_buf_c UsefulBufC |
UsefulBufC and UsefulBuf are simple data structures to hold a pointer and length for a binary data.
In C99 this data structure can be passed on the stack making a lot of code cleaner than carrying around a pointer and length as two parameters.
This is also conducive to secure code practice as the lengths are always carried with the pointer and the convention for handling a pointer and a length is clear.
While it might be possible to write buffer and pointer code more efficiently in some use cases, the thought is that unless there is an extreme need for performance (e.g., you are building a gigabit-per-second IP router), it is probably better to have cleaner code you can be most certain about the security of.
The non-const UsefulBuf is usually used to refer a buffer to be filled in. The length is the size of the buffer.
The const UsefulBufC is usually used to refer to some data that has been filled in. The length is amount of valid data pointed to.
A common use is to pass a UsefulBuf to a function, the function fills it in, the function returns a UsefulBufC. The pointer is the same in both.
A UsefulBuf is NULL, it has no value, when the ptr in it is NULL.
There are utility functions for the following:
See also UsefulOutBuf. It is a richer structure that has both the size of the valid data and the size of the buffer.
UsefulBuf is only 16 or 8 bytes on a 64- or 32-bit machine so it can go on the stack and be a function parameter or return value.
UsefulBuf is kind of like the Useful Pot Pooh gave Eeyore on his birthday. Eeyore's balloon fits beautifully, "it goes in and out like anything".
| typedef struct useful_buf UsefulBuf |
The non-const UsefulBuf typically used for some allocated memory that is to be filled in.
The len is the amount of memory, not the length of the valid data in the buffer.
| typedef struct useful_out_buf UsefulOutBuf |
UsefulOutBuf is a structure and functions (an object) that are good for serializing data into a buffer such as is often done with network protocols or data written to files.
The main idea is that all the pointer manipulation for adding data is done by UsefulOutBuf functions so the caller doesn't have to do any. All the pointer manipulation is centralized here. This code will have been reviewed and written carefully so it spares the caller of much of this work and results in much safer code with much less work.
The functions to add data to the output buffer always check the length and will never write off the end of the output buffer. If an attempt to add data that will not fit is made, an internal error flag will be set and further attempts to add data will not do anything.
Basically, if you initialized with the correct buffer, there is no way to ever write off the end of that buffer when calling the Add and Insert functions here.
The functions to add data do not return an error. The working model is that the caller just makes all the calls to add data without any error checking on each one. The error is instead checked after all the data is added when the result is to be used. This makes the caller's code cleaner.
There is a utility function to get the error status anytime along the way if the caller wants. There are functions to see how much room is left and see if some data will fit too, but their use is generally not necessary.
The general call flow is like this:
UsefulOutBuf can be initialized with just a buffer length by passing NULL as the pointer to the output buffer. This is useful if you want to go through the whole serialization process to either see if it will fit into a given buffer or compute the size of the buffer needed. Pass a very large buffer size when calling Init, if you want just to compute the size.
Some inexpensive simple sanity checks are performed before every data addition to guard against use of an uninitialized or corrupted UsefulOutBuf.
This has been used to create a CBOR encoder. The CBOR encoder has almost no pointer manipulation in it, is much easier to read, and easier to review.
A UsefulOutBuf is 27 bytes or 15 bytes on 64- or 32-bit machines so it can go on the stack or be a C99 function parameter.
| UsefulBufC UsefulBuf_CopyOffset | ( | UsefulBuf | Dest, |
| size_t | uOffset, | ||
| const UsefulBufC | Src | ||
| ) |
Copy one UsefulBuf into another at an offset.
| [in] | Dest | Destiation buffer to copy into |
| [in] | uOffset | The byte offset in Dest at which to copy to |
| [in] | Src | The bytes to copy |
This fails and returns NULLUsefulBufC Src.len + uOffset > Dest.len.
Like memcpy, there is no check for NULL. If NULL is passed this will crash.
There is an assumption that there is valid data in Dest up to uOffset as the resulting UsefulBufC returned starts at the beginning of Dest and goes to Src.len + uOffset.
| int UsefulBuf_Compare | ( | const UsefulBufC | UB1, |
| const UsefulBufC | UB2 | ||
| ) |
Compare two UsefulBufCs.
| [in] | UB1 | The destination buffer to copy into |
| [in] | UB2 | The source to copy from |
Returns a negative value if UB1 if is less than UB2. UB1 is less than UB2 if it is shorter or the first byte that is not the same is less.
Returns 0 if the UsefulBufs are the same.
Returns a positive value if UB2 is less than UB1.
All that is of significance is that the result is positive, negative or 0. (This doesn't return the difference between the first non-matching byte like memcmp).
| size_t UsefulBuf_FindBytes | ( | UsefulBufC | BytesToSearch, |
| UsefulBufC | BytesToFind | ||
| ) |
Find one UsefulBuf in another.
| [in] | BytesToSearch | UsefulBuf to search through |
| [in] | BytesToFind | UsefulBuf with bytes to be found |
| void UsefulOutBuf_Init | ( | UsefulOutBuf * | me, |
| UsefulBuf | Storage | ||
| ) |
Initialize and supply the actual output buffer.
| [out] | me | The UsefulOutBuf to initialize |
| [in] | Storage | Buffer to output into |
Intializes the UsefulOutBuf with storage. Sets the current position to the beginning of the buffer clears the error.
This must be called before the UsefulOutBuf is used.
| void UsefulOutBuf_InsertUsefulBuf | ( | UsefulOutBuf * | me, |
| UsefulBufC | NewData, | ||
| size_t | uPos | ||
| ) |
Inserts bytes into the UsefulOutBuf.
| [in] | me | Pointer to the UsefulOutBuf |
| [in] | NewData | UsefulBuf with the bytes to insert |
| [in] | uPos | Index in output buffer at which to insert |
NewData is the pointer and length for the bytes to be added to the output buffer. There must be room in the output buffer for all of NewData or an error will occur.
The insertion point must be between 0 and the current valid data. If not an error will occur. Appending data to the output buffer is achieved by inserting at the end of the valid data. This can be retrieved by calling UsefulOutBuf_GetEndPosition().
When insertion is performed, the bytes between the insertion point and the end of data previously added to the output buffer is slid to the right to make room for the new data.
Overlapping buffers are OK. NewData can point to data in the output buffer.
If an error occurs an error state is set in the UsefulOutBuf. No error is returned. All subsequent attempts to add data will do nothing.
Call UsefulOutBuf_GetError() to find out if there is an error. This is usually not needed until all additions of data are complete.
| UsefulBufC UsefulOutBuf_OutUBuf | ( | UsefulOutBuf * | me | ) |
Returns the resulting valid data in a UsefulOutBuf.
| [in] | me | Pointer to the UsefulOutBuf. |
The storage for the returned data is Storage parameter passed to UsefulOutBuf_Init(). See also UsefulOutBuf_CopyOut().
This can be called anytime and many times to get intermediate results. It doesn't change the data or reset the current position so you can keep adding data.
| UsefulBufC UsefulOutBuf_CopyOut | ( | UsefulOutBuf * | me, |
| UsefulBuf | Dest | ||
| ) |
Copies the valid data out into a supplied buffer.
| [in] | me | Pointer to the UsefulOutBuf |
| [out] | Dest | The destination buffer to copy into |
This is the same as UsefulOutBuf_OutUBuf() except it copies the data.
| const void * UsefulInputBuf_GetBytes | ( | UsefulInputBuf * | me, |
| size_t | uNum | ||
| ) |
Get pointer to bytes out of the input buffer.
| [in] | me | Pointer to the UsefulInputBuf. |
| [in] | uNum | Number of bytes to get |
This consumes n bytes from the input buffer. It returns a pointer to the start of the n bytes.
If there are not n bytes in the input buffer, NULL will be returned and an error will be set.
It advances the current position by n bytes.
1.9.5