3.12. Numeric Precision

So far we have referred to only the Integer and Real numeric types. In fact, in Object Pascal there are several other numeric types (this is also true in other languages like C and C++). One might ask why there is not simply a single numeric type for all numbers since, after all, integers are real numbers. The reason is efficiency both in terms of memory and processor instructions. Remember that a byte consists of 8 bits. On 32-bit processors, this means that the processor can execute instructions on 4 byte quantities at a time. What does 4 bytes mean in terms of the largest number that can be held? Start with thinking about the numbers that can be represented by 1 bit: 0 or 1. So 1 bit can represent 2 numbers. Now what about 2 bits: 00, 01, 10, 11. These are four binary numbers. If we convert to decimal these are the numbers 0, 1, 2, and 3. Do you see a pattern? 1 bit represents numbers, 2 bits represents numbers, 3 bits represents numbers, and so on. So 32 bits can represent the numbers from 0 to . But what about negative numbers? Since a number can be only either positive or negative, 1 bit must be taken up to represent the sign of a number. Thus, if we want to hold a signed quantity in 32-bits we can represent negative numbers and positive numbers. Since 0 takes up one of these numbers, 32-bit signed quantities can range from -2147483648 to 2147483647. By the way, the Integer type that we have been using in Object Pascal is currently defined as a 32-bit signed quantity. 32-bit processors are optimized to deal with numbers that are 32-bits, 16-bits, and 8-bits (powers of 2). Therefore, while we could always declare variables as Real types, if we know that we're using only whole numbers between -2147483648 to 2147483647, we can improve the performance of our program (in terms of speed) by using Integer types. Object Pascal provides types that also correspond to unsigned quantities. This is useful if we need only values between 0 and 4294967296.

A 32-bit processor deals with 32-bit, 16-bit, and 8-bit quantities with near equal efficiency (32-bit is slightly more efficient). So, why bother with the smaller 16-bit and 8-bit types if 32-bit is larger and more efficient in terms of speed. The answer is memory. While the processor does not care, the file size on a hard disk or in RAM depends on the choice of numeric type. For example, an 8-bit unsigned quantity can hold values from 0 to 255 (). If you know that the data you want to store will never be larger than 255, then there is no reason to store it in a 32-bit integer and waste 3 bytes. This is inconsequential when dealing with one or two variables, but think of what happens in an array with 1000 elements. If the elements are 32-bits each when you only needed 8-bits, you have wasted 24,000 bits (3,000 bytes or about 2.9kB). Of course, modern computers have nearly limitless memories so 2.9kB is almost meaningless today (remember, the Pascal language was born in the 70s when memory was extremely limited). So it's not something to obsess about for most small applications, but memory overhead can easily turn a very large application with thousands of variables into a bulky tortoise if one is not careful.

The following table lists the integer types and sizes in Object Pascal (most other languages like C have similar types). The column Fixed says whether or not the size and range of the type may change in the future. As a general rule, you should almost always use Integer for all whole numbers unless memory is at a premium or if you are reading a file that requires a certain numeric size. Integer is the name given to the signed type that most processors deal with efficiently (as of now that's 32-bit though it may change as 64-bit processors become more prevelant). If you need to always have an exact size for a quantity you should use one of the values for which Fixed is Yes in the table below.

The story is similar for the Real type. Unlike the Integer types, coding for the real types has to obey the same rules for the number of values that can be encoded in N-bits. If we have a 32-bit value, it can represent no more than 4294967295 values. However, the story is slightly more complex because we are dealing with decimal values and not whole numbers. For example, a 32-bit real has only 7-8 significant digits (by comparison, 4294967295 has 10 significant digits). Remember that for signed integers, 1-bit of the 32 bits had to be taken for the sign. In real numbers, some of the bits are taken to determine the order of magnitude. This is a tricky concept to understand at first, but a few examples will help clarify. Say we want to create a numeric variable that can hold the number 12 followed by 40 zeros. Remember that a 32-bit unsigned value holds numbers around 4 billion and a 64-bit unsigned value holds numbers up to around so we are clearly out of luck with those choices. However, a real type stores value and order of magnitude separately. So all we have to do is use the following code:

The reason we can do this is that we don't care about the 40 zeros. They are not significant. All that matters is the 12 (the 2 significant digits) and the fact that this is a really big number. If you think about it, there are very few times that one would actually need more than just a few significant digits (and there is almost never any reason to have 40 of them). So 32-bit real types allow us to store very large (or very small... think 12e-40) numbers given that we are happy with with precision to 7-8 significant figures. There is one very important caveat to this statement, however. If one is performing many sequential calculations with real numbers, it is important to use around twice the number of significant digits that one requires for a given application. The reason for this is round-off errors. Multiple calculations with imprecise numbers tend to build up over time and yield erroneous results. For this reason, it is standard to use a 64-bit real type which has 15-16 significant figures when performing mathematical calculations. For extremely precise calculations Object Pascal even provides an 80-bit real type with 19-20 significant figures, but this only consumes additional memory and should not be used for storage unless really necessary. The following table summarizes the real types in Object Pascal:

Note that these ranges do not make any logical sense like the ranges for integer types without more in depth background on how real numbers are actually coded in memory. Another thing to remember about real types is that we mentioned before that the processor is optimized to handle 32-bit integers. Mathematical operations on reals are handled separately by the floating point unit (FPU) of the CPU. The FPU is generally much slower when compared to operations on integers. The reason for this is that integers can be stored at the level of hardware exactly as binary numbers (i.e., there are circuits that are at high and low voltages which represent 0's or 1's). Very simple circuits exist that can compute mathematical operations (like addition) extremely efficiently for binary numbers. Real numbers are coded as Binary Coded Decimals or BCDs (we won't discuss the actual details of what this means because it is beyond the scope of this Unit). Because BCDs are more complex than simple binary integers, more sophisticated circuits are required to operate on them which results in slower operations.