         CRC Theory, by Paul Edwards
         This document is released to the Public Domain
         

CRCs are formally defined in terms of a "generating polynomial", e.g.
the generating polynomial for the CCITT CRC is x^16+x^12+x^5+1.  This,
as you can tell, is simply a load of gibberish, of no interest to us
programmers.  Programmers only use it to make it look like they know
complicated things.  Ask them how that polynomial was worked out in the
first place.  All you need to know, is that mathematicians have worked
out some great polynomial, and this is the form you will see it in.

Now what you must do is immediately convert this into a string of bits,
e.g. the above polynomial becomes 1 0001 0000 0010 0001.  Then you
discared that first 1, to get 0001 0000 0010 0001.  Then you reverse
that pattern to get 1000 0100 0000 1000.  In hex, this is 8408.  
Hardware types will then call this the feedback factor.  Forget that
too.  What you need to know is this (you may as well consider this to
be the formal definition of a CRC):

To compute a new crc (newcrc), given an old crc (oldcrc) and 
a new byte (byte), you use the following algorithm:

   newcrc = oldcrc;
   for (x=0; x < 8; x++)
   {
     if (((newcrc ^ byte) & 0x01) != 0)
     {
       newcrc = newcrc >> 1;
       newcrc = newcrc ^ f;  /* where f is the hex value computed above */
     }
     else
     {
       newcrc = newcrc >> 1;
     }
     byte = byte >> 1;
   }


This is as easy-to-understand definition of a crc that you are ever
likely to encounter.  But while we're here, let's look at an alternative
way of doing the above algorithm.  Instead of XORing each bit of the
byte with the CRC one bit at a time, let's simply XOR the whole byte.

   newcrc = oldcrc;
   newcrc = newcrc ^ byte;
   for (x=0; x < 8; x++)
   {
     if ((newcrc & 0x01) != 0)
     {
       newcrc = newcrc >> 1;
       newcrc = newcrc ^ f;  /* where f is the hex value computed above */
     }
     else
     {
       newcrc = newcrc >> 1;
     }
   }

So where did all those tables come from?  

Well that is a lot more complicated, so take the rest of this very
slowly.  Using the above algorithm, look at the following CRC:

CF CE CD CC CB CA C9 C8 C7 C6 C5 C4 C3 C2 C1 C0

and byte B7 B6 B5 B4 B3 B2 B1 B0.

Now, first of all we XOR B0 with C0.  Let's call the result X0.  If X0
is 1, then we need to XOR f (the hex value from above).  If it is 0, then
we don't XOR.  In both cases we do a shift right.  Then we will be XORing
C1 with B1.  The result X1 either causes an XOR or it doesn't.  Now when
we are about to compute a new CRC, once we get the old CRC and the new
byte, the rest is totally predictable.  If you XOR the low byte of the CRC
with the incoming byte, to get

X7 X6 X5 X4 X3 X2 X1 X0     (we'll call it X for short),

then the rest can be predicted.  E.g., if the number is 

1  0  1  1  0  1  0  1

then the 1 will cause a right shift, followed by an XOR, to give:

00 CF CE CD CC CB CA C9 C8 X7 X6 X5 X4 X3 X2 X1
X0             X0                   X0

Notice how the XORs went in at positions corresponding to 0x8408.  The
above diagram means that at bit position 10, you will find the value of
CB XORed with X0.  C7 to C1 have already been replaced by X7 to X1, since
it is easier to deal with the whole byte as in the second form of the
above CRC algorithm.

Now the next bit, X1, is equal to 0, so all we do is the shift right.
So now we have:

00 00 CF CE CD CC CB CA C9 C8 X7 X6 X5 X4 X3 X2
   X0             X0                   X0
   
The next bit, X2 is a 1, so we get:

00 00 00 CF CE CD CC CB CA C9 C8 X7 X6 X5 X4 X3
X2    X0             X0                   X0
            
The next bit, X3 is a 0, so we get:

00 00 00 00 CF CE CD CC CB CA C9 C8 X7 X6 X5 X4
   X2    X0             X0                   X0
   
Now although X4 was a 1, the new bit to be processed is actually X0 ^ X4,
which is now 1 ^ 1, which is 0.  So we only do the shift this round, not
the XOR, to get:

00 00 00 00 00 CF CE CD CC CB CA C9 C8 X7 X6 X5
      X2    X0             X0   
      
As you can see, what we are going to end up with, is CF to C8 moved down
8 bits UNCHANGED, XORed with some value, which is dependent ONLY on the
original values of X7 to X0 (X for short).  Now X, being only 8 bits, can
only have 256 different values.  If we can compute these in advance, then
the new algorithm for determining the new CRC value is:

X = byte ^ (oldcrc & 0xff)
newcrc = (oldcrc >> 8) ^ table[X]

If you take a break and go and look at all the supplied crc??_update()
functions, you will find that all of them look remarkably similar to this!

Here is a replica of the crcit_update() macro:

#define crcit_update(crc, ch) \
    ((crc) = ((crc) >> 8) ^ crcit_tab[((ch) ^ (crc)) & 0xffU])


Now all we need to do is construct that elusive table!  

The table is meant to convert the value of X into some XOR value.  We
aren't interested in the high-order (CF to C8) values, so we will leave
them as 0.  And we already know what we want the low-order ones to be,
X7 to X0!  So if we want to know the XOR value corresponding to an X of
0x3a, then what we need to do is set the CRC to a value of 0x003a, and 
then cycle through the second algorithm.  The end result will be the
table entry.  Since we want to do this for values of X from 0 to 255,
here is the required algorithm:


   for (i=0; i<256; i++)
   {
     newcrc = i;
     for (x=0; x < 8; x++)
     {
       if ((newcrc & 0x01) != 0)
       {
         newcrc = newcrc >> 1;
         newcrc = newcrc ^ f;  /* where f is the hex value computed above */
       }
       else
       {
         newcrc = newcrc >> 1;
       }
     }
     table[i] = newcrc;
   }


Now, why don't we leave here?  We have everything we need, don't we?  The
simple answer is, YES!  If you are planning on using the table-driven
algorithms, which you probably are, then you can take this and run like
hell.  However, why don't we make life difficult for ourselves?  There are
some uses for what follows, namely when you can't afford to keep a whole
table in memory (ie all 512 bytes of it).  Here it is:

If you go back to where we were showing how the new crc was calculated,
bit by bit, then, since we're only trying to construct the table entry,
we can make the previous CRC value 0, we have the following:

00 00 00 00 00 00 00 00 X7 X6 X5 X4 X3 X2 X1 X0

Now if the value of X0 is 1, we need to XOR in f, but if it is 0, then 
we don't.  If f is 0x8408, we XOR in 8408, right?  That is, we XOR in

01 00 00 00 00 01 00 00 00 00 00 00 01 00 00 00

Instead of this, we can XOR in

X0 00 00 00 00 X0 00 00 00 00 00 00 X0 00 00 00  

REGARDLESS OF WHETHER X0 is 1 or 0.  Why is this?  Because if X0 is 1,
it gets XORed in as required, but if X0 is 0, the XOR has no effect
anyway, so doesn't cause any harm!

So now we have:

00 00 00 00 00 00 00 00 00 X7 X6 X5 X4 X3 X2 X1
X0             X0                   X0

And after the next round, we get

00 00 00 00 00 00 00 00 00 00 X7 X6 X5 X4 X3 X2 
   X0             X0                   X0
X1             X1                   X1

Then, and we will start combining rows where possible now:

00 00 00 00 00 00 00 00 00 00 00 X7 X6 X5 X4 X3  
X2 X1 X0       X2 X1 X0             X2 X1 X0

Next is:

00 00 00 00 00 00 00 00 00 00 00 00 X7 X6 X5 X4
X3 X2 X1 X0    X3 X2 X1 X0          X3 X2 X1 X0

Then:

00 00 00 00 00 00 00 00 00 00 00 00 00 X7 X6 X5
X4 X3 X2 X1 X0 X4 X3 X2 X1 X0       X4 X3 X2 X1
X0             X0                   X0

Notice how X0 has done a full-circle, and is being propogated for the
second time.  For the next one we have to start splitting the rows up
again, so that you can see a nice pattern emerging:

00 00 00 00 00 00 00 00 00 00 00 00 00 00 X7 X6
X5 X4 X3 X2 X1 X0                   X5 X4 X3 X2
               X5 X4 X3 X2 X1 X0
X1 X0          X1 X0                X1 X0

Then:

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 X7
X6 X5 X4 X3 X2 X1 X0                X6 X5 X4 X3
               X6 X5 X4 X3 X2 X1 X0
X2 X1 X0       X2 X1 X0             X2 X1 X0

And finally:

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
X7 X6 X5 X4 X3 X2 X1 X0             X7 X6 X5 X4
               X7 X6 X5 X4 X3 X2 X1 X0
X3 X2 X1 X0    X3 X2 X1 X0          X3 X2 X1 X0


As you can see, for any value X that we are trying to compute the table
entry for, it ends up being a couple of shifts of the complete byte,
followed by 2 different partial bytes.  Also, you can see that these
rows overlap.  So, for the CCITT CRC, the algorithm to calculate the
table entry goes like this:

  a = X & 0x0f;
  b = X ^ (a << 4);
  entry = (b << 8) ^ (b << 3) ^ (b >> 4);


That is the end of the heavy stuff.  The rest of this document is just
"trivial" implementation details.  The CRC-16, CCITT CRC and the Xmodem
CRC all start of with an initial CRC value of 0.  However, the CRC-32
starts off with it equal to 0xFFFFFFFF.  

In the above diagrams, you always see the CRC shifting right.  Well, the 
Xmodem one is an exception.  It is the same as the CCITT CRC one, except 
that it shifts to the left, so the algorithms are twisted around a bit.

Also, it seems that the CCITT CRC tables can be used for Xmodem, so long
as the macro is changed and you pass two 0's through the update macro
when you are about to end.  I didn't bother investigating this.


A word of warning.  I am no authority on CRCs, so don't bet your life
savings on the accuracy of this document.  If you find an error in it,
please report it to me.  I can be contacted through the below address:

Paul Edwards
3:711/934.0@fidonet

I run a BBS called the Ten Minute Limit, where you can be sure to get
the unchanged files.
