[PATCH v3 1/5] CRC64 commands documentation
Ulf Samuelsson
binutils@emagii.com
Sun Feb 19 17:38:39 GMT 2023
Hmm, the comments did not arrive. Maybe just delayed.
Here they are:
=========================================
Here is the third patchset for introducing CRC64 generation.
This has focused on supporting maintainability of the feature.
In order to do so, CRC32 generation has been introduced to
detect issues when extending the linker with another algoritm
for checking integrity of the image.
This will simplify adding other algorithms like SHA algoritms
in the future.
In addition, the TIMESTAMP command is used to store the current time
in seconds since Epoch is stored in the image.
The main routine for calculating CRCs has been broken down
in subroutines which allow other algorithms.
An algorithm typically needs
<type> *init_<algo>_tab( <type> poly )
<type> calc_<algo>_inv( const unsigned char *input_str, size_t num_bytes );
<type> calc_<algo> ( const unsigned char *input_str, size_t num_bytes );
void lang_add_<algo>_syndrome(bool invert, bfd_vma poly)
void lang_add_<algo>_table(void)
The common functions are:
* bool get_text_section_contents(void)
This reads the '.text' section and its contents
Pointers are stored in the global variables
asection *text_section
char *text_contents
* bool set_crc_checksum(bfd_vma crc, bfd_vma addr, bfd_vma size)
Stores the CRC at the index addr. The size of the CRC in bytes is
specified.
Storing something larger that 64-bit is not supported here.
* bool symbol_lookup(char *name, bfd_vma *val)
Gets the value of a symbol into 'val'. Returns false if not found.
To add CRC32 support means adding the following commands
* CRC32 CRC32 '(' crc_start ',' crc_end ')'
* CRC32 POLY '[' mustbe_exp ']' '(' crc_start ',' crc_end ')'
* CRC32 POLYI '[' mustbe_exp ']' '(' crc_start ',' crc_end ')'
================
Here is the second patchset for introducing CRC64 generation in the linker
This is mainly looking at indentation and whitespace errors.
The patches applies cleanly without whitespace error
except for the last patch which contains the testsuite.
When the contents of .text is printed out, sometimes
the last byte of a line in the CRC table is a space.
This is reported as a whitespace error.
Supports the following new linker commands:
* CRC64 ECMA '(' crc_start ',' crc_end ')'
* CRC64 ISO '(' crc_start ',' crc_end ')'
* CRC64 POLY '[' mustbe_exp ']' '(' crc_start ',' crc_end ')'
* CRC64 POLYI '[' mustbe_exp ']' '(' crc_start ',' crc_end ')'
* CRC64 TABLE
ECMA == 0x42F0E1EBA9EA3693
ISO == 0xD800000000000000
POLY == Allows your own polynome
POLYI == Allows your own polynome, with inversion during calc
The CRC is calculated from crc_start to crc_end (not included)
The "CRC64 <polynome> command
* Allows specifying the polynom (ECMA(default), ISO or your own)
* Allows for inversion in the CRC calculation (CRC64-WE)
* Allows specifying the area that should be checked.
* reserves room for the CRC (8 bytes)
* Declares a symbol ___CRC64___ for the address of the CRC.
* Declares a symbol ___CRC64_START___ for the first address of the
checked area
* Declares a symbol ___CRC64_END___ for the first address after the
checked area
The symbols names are declared in "checksum.h".
If the names are unsuitable, it is easy to change.
The CRC TABLE command
This is used to speed up the CRC calculation.
* Creates a 2kB table which speeds up the CRC calculation
* Can insert the 2kB table into the .text section at current location.
* Declares a symbol ___CRC64_TABLE___ if the table is inserted.
Comments on the code:
The process of calculation involves:
* Check if CRC is requested
* Validation of CRC prerequisites
* get .text section
* get .text section contents
* calculate CRC over the area
* insert the CRC in the correct location
====================================
This version also supports the
* DEBUG ON
* DEBUG OFF
turning on/off yydebug
I find it useful for debugging the build, but it can easly be removed.
This patch contains a lot of debug output, that is enabled by
#define DEBUG_CRC 1
This should be removed in the final version.
The ld.texi stuff needs some more work. Not very experienced with that.
Added an entry in NEWS
Added 4 test cases for the different CRC64 polynome commands.
All testcases generate a CRC table.
The code is using the libcrc released under an MIT license found in
* https://www.libcrc.org/
* https://github.com/lammertb/libcrc/tree/v2.0
Author: Lammert Bies
A license for libcrc is added to the patch.
Best Regards
Ulf Samuelsson
Den 2023-02-19 kl. 18:34, skrev binutils@emagii.com:
> From: Ulf Samuelsson <binutils@emagii.com>
>
> * LIBCRC license
> * NEWS
> * ls.texi
>
> Signed-off-by: Ulf Samuelsson <binutils@emagii.com>
> ---
> COPYING.CRC64 | 42 +++++++++++++++
> ld/NEWS | 47 +++++++++++++++++
> ld/ld.texi | 140 ++++++++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 229 insertions(+)
> create mode 100755 COPYING.CRC64
>
> diff --git a/COPYING.CRC64 b/COPYING.CRC64
> new file mode 100755
> index 00000000000..618e1c63dae
> --- /dev/null
> +++ b/COPYING.CRC64
> @@ -0,0 +1,42 @@
> +The GNU linker contains CRC routines that are used to implement the
> +CRC64 command in the output section.
> +
> +The CRC routines are extracted from LIBCRC available at
> +
> +They are used to
> +* https://www.libcrc.org/
> +* https://github.com/lammertb/libcrc/tree/v2.0
> +
> +
> +/*
> + * Library: libcrc
> + * File: src/crc64.c
> + * Author: Lammert Bies
> + *
> + * This file is licensed under the MIT License as stated below
> + *
> + * Copyright (c) 2016 Lammert Bies
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a copy
> + * of this software and associated documentation files (the "Software"), to deal
> + * in the Software without restriction, including without limitation the rights
> + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> + * copies of the Software, and to permit persons to whom the Software is
> + * furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in all
> + * copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> + * SOFTWARE.
> + *
> + * Description
> + * -----------
> + * The source file src/crc64.c contains the routines which are needed to
> + * calculate a 64 bit CRC value of a sequence of bytes.
> + */
> diff --git a/ld/NEWS b/ld/NEWS
> index 4b91f2c3b0a..8c1203a3c2e 100644
> --- a/ld/NEWS
> +++ b/ld/NEWS
> @@ -1,5 +1,52 @@
> -*- text -*-
>
> +* The linker script has a new command to insert a timestamp
> + TIMESTAMP
> + inserts the current time (seconds since Epoch) as a 64-bit value
> +
> +* The linker script syntax has new commands for debugging a linker script
> + DEBUG ON turns on debugging
> + DEBUG OFF turns off debugging
> +
> +* The linker script syntax has new commands for handling CRC-32/64 calculations
> + on the '.text' section
> + It uses code from https://www.libcrc.org/
> +
> + CRC32 CRC32 (start, end)
> + CRC32 POLY (polynome)(start, end)
> + CRC32 POLYI(polynome)(start, end) (inversion during CRC calculation)
> + CRC64 ECMA (start, end)
> + CRC64 ISO (start, end)
> + CRC64 POLY (polynome)(start, end)
> + CRC64 POLYI(polynome)(start, end) (inversion during CRC calculation)
> + CRC64 TABLE
> +
> + The CRC32, ECMA, ISO, POLY and POLYI defines the polynome to use
> + and reserves space for the 32/64-bit CRC in the '.text' section.
> + The default polynome is the ECMA.
> +
> + The CRC32 <polynome> command defines some global symbols.
> + ___CRC32___ is the address of the CRC64 checksum
> + ___CRC32_START___ is the address where CRC calculation starts
> + ___CRC32_END___ The CRC calculation ends before this address.
> +
> + The CRC64 <polynome> command defines some global symbols.
> + ___CRC64___ is the address of the CRC64 checksum
> + ___CRC64_START___ is the address where CRC calculation starts
> + ___CRC64_END___ The CRC calculation ends before this address.
> +
> + All three symbols must refer to addresses in the '.text' section.
> + If they are defined at the end of the linking process, then
> + the CRC## will be calculated between ___CRC##_START___ .. ___CRC##_END___ -1
> + and the result will be inserted at ___CRC##___.
> + ___CRC##___ must be outside the region where CRC is calculated
> +
> + The CRC32/64 TABLE command generates a table for CRC generation.
> + This is not neccessary, but will speed up the calculation
> + and makes it compatible with the CRC check routines at https://www.libcrc.org/
> + It defines the ___CRC##_TABLE___ symbol at the start of the table.
> + The user may choose to add this table to his code instead of using the linker.
> +
> * The linker script syntax has two new commands for inserting text into output
> sections:
> ASCII (<size>) "string"
> diff --git a/ld/ld.texi b/ld/ld.texi
> index 7802f0661b0..2042fadb33e 100644
> --- a/ld/ld.texi
> +++ b/ld/ld.texi
> @@ -5394,6 +5394,146 @@ area:
> ASCIZ "This is 16 bytes"
> @end smallexample
>
> +@cindex output section strings
> +@kindex CRC32 CRC32 (@var{expression}, @var{expression})
> +@kindex CRC32 POLY (@var{expression})(@var{expression}, @var{expression})
> +@kindex CRC32 POLYI (@var{expression})(@var{expression}, @var{expression})
> +@kindex CRC64 ECMA (@var{expression}, @var{expression})
> +@kindex CRC64 ISO (@var{expression}, @var{expression})
> +@kindex CRC64 POLY (@var{expression})(@var{expression}, @var{expression})
> +@kindex CRC64 POLYI (@var{expression})(@var{expression}, @var{expression})
> +
> +You can calculate the CRC of a part of the '.text' section through the
> +@code{CRC32} and @code{CRC64} commands.
> +The commands take a parameter for the @code{polynome} and then two more,
> +specifying range of the checked area. The end address is the first address
> +past the checked area.
> +
> +There is one predefined 43-bit polynomes
> +
> +* @code{CRC32} @code{0xEDB88320}
> +
> +There are two predefined 64-bit polynomes
> +
> +* @code{ECMA} @code{0x42F0E1EBA9EA3693}
> +
> +* @code{ISO} @code{0xD800000000000000}
> +
> +You can also select your own @code{polynome} using the @code{CRCxx POLY} or
> +@code{CRCxx POLYI}. The @code{POLYI} will invert the polynome before and after
> +the calculation which is neccessary for the @code{CRC64-WE} algorithm
> +
> +The default 64-bit polynome is the @code{ECMA}
> +
> +The CRC32 <polynome> command defines some global symbols.
> +
> +* @code{___CRC32___} address of the CRC32 checksum
> +
> +* @code{___CRC32_START___} first address in the checked area.
> +
> +* @code{___CRC32_END___} first address past the checked area.
> +
> +The CRC64 <polynome> command defines some global symbols.
> +
> +* @code{___CRC64___} address of the CRC64 checksum
> +
> +* @code{___CRC64_START___} first address in the checked area.
> +
> +* @code{___CRC64_END___} first address past the checked area.
> +
> +Note: The generated CRC value must be stored outside the checked area.
> +
> +Example 1: This request a CRC check using the @code{ECMA} algorithm
> +
> +@smallexample
> + CRC = '.';
> + CRC64 ECMA (START_CHECK,END_TEXT)
> + START_CHECK = '.';
> +@end smallexample
> +
> +The user can retrieve the CRC value through the @code{CRC} label.
> +
> +Example 2: This request a CRC check using the @code{ISO} algorithm
> +
> +@smallexample
> + CRC = '.';
> + CRC64 ISO (START_CHECK,END_TEXT)
> + START_CHECK = '.';
> +@end smallexample
> +
> +The user can retrieve the CRC value through the @code{CRC} label.
> +
> +Example 3: This request a CRC check using a user defined @code{polynome}
> +
> +@smallexample
> + CRC = '.';
> + CRC64 POLY (0xDEADBEEFDEADBEEF) (START_CHECK,END_TEXT)
> + START_CHECK = '.';
> +@end smallexample
> +
> +The user can retrieve the CRC value through the @code{CRC} label.
> +
> +Example 4: This request a CRC check using a user defined @code{polynome}
> + The @code{polynome} is inverted before use, and teh result is inverted.
> +
> +@smallexample
> + CRC = '.';
> + CRC64 POLYI (0xDEADBEEFDEADBEEF) (START_CHECK,END_TEXT)
> + START_CHECK = '.';
> +@end smallexample
> +
> +Example 5: This request a CRC check using a user defined @code{CRC32}
> +
> +@smallexample
> + CRC = '.';
> + CRC32 CRC32 (START_CHECK,END_TEXT)
> + START_CHECK = '.';
> +@end smallexample
> +
> +The user can retrieve the CRC value through the @code{CRC} label.
> +
> +@cindex output section strings
> +@kindex CRC32 TABLE
> +
> +The @code{CRC32 TABLE} command creates a 1 kByte table for a table-driven
> +CRC calculation. This speeds up the CRC calculation over a non-table-driver
> +version since you can handle 8 bits at a time, instead of 1 bit.
> +
> +The table generated is for the @code{polynome} selected using a @code{CRC32}
> +command.
> +
> +Example 1: Generate a 1 kB table
> +@smallexample
> + mytable = '.';
> + CRC32 TABLE
> +@end smallexample
> +
> +The user can declare @code{extern uint32_t *mytable;} in his code to use it.
> +
> +@cindex output section strings
> +@kindex CRC64 TABLE
> +
> +The @code{CRC64 TABLE} command creates a 2 kByte table for a table-driven
> +CRC calculation. This speeds up the CRC calculation over a non-table-driver
> +version since you can handle 8 bits at a time, instead of 1 bit.
> +
> +The table generated is for the @code{polynome} selected using a @code{CRC64}
> +command. If no @code{CRC64} command has been emitted, the @code{ECMA} is used.
> +
> +Example 1: Generate a 2 kB table
> +@smallexample
> + mytable = '.';
> + CRC64 TABLE
> +@end smallexample
> +
> +The user can declare @code{extern uint64_t *mytable;} in his code to use it.
> +
> +@cindex output section strings
> +@kindex TIMESTAMP
> +
> +The @code{TIMESTAMP} command creates 64-bit integer with the number of seconds
> +since Epoch (1970-01-01 00:00)
> +
> @kindex FILL(@var{expression})
> @cindex holes, filling
> @cindex unspecified memory
More information about the Binutils
mailing list