Historical Background

When Isaac Newton and Gottfried Wilhelm Leibniz first formulated differential calculus they effectively made use of the concept of an infinitesimal, which they referred to as an infinitely small number, whatever that was supposed to mean. For example, Leibniz, who introduced the d notation for differentials, said in deriving d(xy)=xdy+ydx that this follows from d(xy) = (x+dx)(y+dy) − xy = xdy + ydx + dxdy and the omission of the quantity dxdy, which is infinitely small in comparison with the rest, for it is supposed that dx and dy are infinitely small.

In effect he was saying that the although infinitesimals are not zero the product of infinitesimals is zero.

Newton's analysis involved taking ratios of infinitesimals. Those terms for the ratio that which had an infinitesimal as a factor were equated to zero, or as he expressed it:

[T]erms which have [an infinitesimal] as a factor will be equivalent to nothing in respect to the others. I therefore cast them out…

Clearly the concept of an infinitesimal was pretty fuzzy. Nevertheless researchers found the infinitesimal concept , useful even essential, for developing the differential calculus. In the latter nineteenth century the German mathematician Karl Weierstrauss introduced the epsilon-delta process which provided a rigorous basis for the calculus and mathematics instructors thereafter discouraged students from using the infinitesimal concept. There were however a large number of recalcitrants who kept the comfortable nonsense of infinitesimals alive.

And then in 1960 Abraham Robinson founda way to provide a rigorous foundations for infinitesimals and thus infinitesimals were acceptable, although not exactly welcome, again in mathematical discourse. Robinson called his formulation non-standard analysis. The purpose of this material is to explain, illustrate and justify the non-standard analysis formulation of infinitesimals. It will not utilize exactly Robinson's formulation. Instead it will use a formulation that is handier; i.e., an infinitesimal is a quantity so small that although it is not zero its square and higher powers are zero. No real number can have that property. To satisfy that property a new set of numbers must be created. The situation is analogous to the notion of the square root of negative one. No square of a real number can be negative one so to satisfy this condition a new set of numbers must be created.

The creation of a new set of numbers requires a review of some mathematical foundations.

Mathematical Fields

A mathematical field is a set and two operations defined on the elements of that set, say (S, +, *). The first operation, +, (called addition) is such that:

• it is associative: a + (b + c) = (a + b) + c
• it is commutative: a + b = b + a
• there exists an additive identity, say 0, in S such that for all a belonging to S, a + 0 = a.
• there exist additive inverses, such that for any element a of S, there is a b in S such that a+b = 0.

These conditions amount to (S, +) being an abelian group.

The second operation, *, (called multiplication) is such that:

• it is associative: a*(b*c) = (a*b)*c.
• it is left distributive over +: a*(b+c) = a*b + a*c.
• it is right distributive over +: (b+c)*a = b*a + c*a.
• there exists a multiplicative identity e in S such that for any element of S, e*a = a.
• for any element except the additive identity 0 there exists a multiplicative inverse; i.e., for any a except 0 there is a b in S such that b*a=e.

One example of a field is the rational numbers Q. The rational numbers are usually denoted as a/b where b≠0, but the slash / is merely symbolic. They could be denoted just as well as ordered pairs (a,b). But k*a/k*b is the same element as a/b. Addition and multiplication are defined. The additive identity is 0/1 and multiplicative identity is 1/1. The additive inverse of a/b is -a/b. The multiplicative inverse of a/b for a≠0 is b/a.

Another example of a field is the real numbers R with the usual definition of addition and multiplication. A real number corresponds roughly to an infinite string of digits. This is not precisely the correspondence because 1.999… is the same real number as 2.0000… See real number system for a more complete story on the logical foundations of the real number system.

Extensions of Fields

A new field can be created from a known field by a process of what is called "adjoining an element to the field." For example, the set of elements of the form p+q√2, where p and q are rational numbers is a new field. The definition of the addition of (p1+q1√2) with (p2+q2√2) is [(p1+p2)+(q1+q2)√2]. The definition of the multiplicative product of p1+q1√2 with p2+q2√2 is (p1p2+2q1q2)+(p1q2+p2q1)√2. The additive inverse and the multiplicative inverse are easily defined.

The complex numbers are often thought of as an extension of the reals created by adjoining the imaginary element i, where i2=-1.

This procedure of adjoining an element to a field is a standard way of explaining the extension of a field, but there is something about it that seems suspicious. Clearly, because of the name imaginary chosen for its basic unit, the creation of the complex numbers raised some doubts and suspicions when it was first formulated. The process of adjoining an element seems to hypothesize the existence of something for which there may be considerable doubt about its existence. As will be shown below the problem with process of adjoining of adjoining an element is merely a matter of how it is explained.

Before proceding a few remarks about √2 are in order. We now say that the root of x2=2 is √2 and identify it with 1.41214…, but consider a time when the most general field known was the rational numbers. The Greeks were able to prove that √2 could not be a rational number. It would have appeared at that time that hypothesizing a solution to x2=2 was congering up an impossible entity. Without the Arabic (Indian) notation for numbers the Greeks had no way of representing √2 as anything like 1.414214… Even today with the model of the real numbers being essentially infinite sequences of digits as a model for the real numbers we cannot explicitly write the value of √2. We can write √2 to any degree of approximation we desire but we cannot give √2 in its entirety. Nevertheless the adjunction of the solution to x²=2 creates a field extension for the rational numbers.

However the creation of extensions of a field using formulas of the form a+bγ are not necessary. What is really involved with such a field extension is that ordered pairs of rational numbers are considered with new definitions of the field operations of addition and multiplication. That is to say, a new field is defined in terms of the set of ordered pairs (p,q) and the addition of (p1,q1) and (p2,q2) is (p1+p2,q1+q2). The multiplication of (p1,q1) and (p2,q2) is ((p1p2+2q1q2),(p1q2+p2q1)) With these definitions the product (0,1)*(0,1) is equal to (2,0)=2(1,0). Thus (0,1) is the solution to the equation x²=2.

Likewise there is a similar method for generating the complex numbers. Simply consider a complex number to be an ordered pair of real numbers with the multiplication of (a,b) with (c,d) to be defined as (ac-bd,ad+bc). Thus (0,1)*(0,1)=(-1,0)=-(1,0) so (0,1)2=-(1,0) and hence (0,1) is the square root of -1.

Consider the extension of the field of integers to the rational numbers. If we seek solutions to the equation bx=a (equivalently bx-a=0) the field extension is ordered pairs of integers (a,b). In this extended field a=(a,1) and b=(b,1). Multiplication of (p,q) and (r,s) gives (pr,qs). Thus if x=(a,b) then bx=(b,1)(a,b)=(ba,b)=(a,1)=a. Thus (a,b) is a solution to the equation bx=a. The addition of (p,q) with (r,s) has the more complicated formula ((ps+rq),qs).

Note for each of these examples, there is a subfield of the field extentions which is isomorphic to the original field. In the case of p+q√2 the subfield for all elements such that q=0. For the complex numbers a+b√(-1) the subfield is of all numbers such that b=0. For the extension of the integers to the rational numbers (a/b) the subfield is rational numbers such that b=1. For all extensions of a field there must be some subfield of the extended field which is isomorphic to the original field.

Another term which is applied to this process is creating the field of fractions. This term is used to describe the creation of a field from the more general structure of a ring, a structure like a field but not having multiplicative inverses for its elements..

The Polish mathematician Jan Mikusinski developed an interesting field of mathematics using this construction where the set S is of functions and multiplication is convolution. It is called Operational Calculus.

For more on field extensions see Galois Theory.

Note that this process of adjoining an element to a field tell us nothing about the entities such as √2 or i=√(-1); it just gives us a field in which there is an element which satisfies the defining condition for the entity. Of course the process can be used to create an extension field from the rational numbers in which there is a solution to x²=3 or x²=5 and so forth. (In fact for any polynomial equation root.)

The Nature of Infinitesimals

Considered the defining conditions for an infinitesimal ε as being ε≠0 and ε²=0. If we consider elements of the form a+bε and c+dε, where a,b,c,d are real numbers, the definitions of addition and multiplication are given by (a+c)+(b+d)ε and ac + (ad+bc)ε. These operations can be restated in terms of order pairs as (a,b)+(c,d)=((a+c),(b+d)) and (a,b)*(c,d)=(ac,(ad+bc)). This addition is obviously associative and commutative. There exists the additive identity of (0,0) and the additive inverse of (a,b) is (-a,-b). The associativity of multiplication holds because

((a,b)*(c,d))*(e,f) = (ac,ad+bc)*(e,f) = (ace, acf+bce)
which is the same as
(a,b)*((c,d)*(e,f)) = (a,b)*(ce,cf+de)) = (ace, acf+bce)

The identity for multiplication is (1,0). The problem comes in finding a multiplicative inverse for every element other than (0,0). In particular, what could be the multiplicative inverse of (0,1)?

The mathematical area called differential forms establishes a system in which the square of a differential is zero.

It turns out that in nonstandard analysis this above condition for an infinitesimal is not required. Nonstandard analysis provides a way to extend the real number field to a field, called the hyperreals in which entities exist that have the essential properties of infinitesimals. But an extension to a new field requires that each nonzero element have a multiplicative inverse. This means that a nonzero infinitesimal must have an inverse element. The inverse of an infinitely small entity has to be an infinitely large entity. Since positive infinitesimals are considered to be nonzero entities less than any positive real number the appropriate quantity for an multiplicative inverse would be an entity which is greater than any real number but not equal to infinity, a sort of infinitude. In keeping with the conceptualization of infinitesimals, an infinitude would be a numerical entity so large that while it is not infinite its square is.

Order Relations on Fields

A field is ordered if an ordering can be defined such that for any two elements of the set a and b, either a>b, a=b or b>a. For two rational numbers a/b and c/d their order is determined by the sign of ad-bc; i.e., if ad-bc>0 then (a/b)>(c/d) and so forth.

In addition to the condition expressed above, called the trichotomy condition, The order relation must satisfy the following conditions:

• transitivity: if a>b and b>c then a>c
• summation: for all a, b and c in S, if a>b then (a+c)>(b+c)
• product: for all a, b and c in S, if a>b then (a*c)>(b*c)

For some purposes it is more convenient to utilize the order relation ≥. In this formulation equality of a and b means a≥b and b≥a.

The Completeness of an Ordered Field

An ordered field is complete if every one of its nonempty subsets which has an upper bound with respect to the order relation in the field has a least upper bound with respect to that order relation.

While there are many different fields, there is essentially only one complete ordered field, the real numbers, in the sense that any complete ordered field is isomorphic to the real numbers.

On the Existence of an Infinitesimal

There is a theorem that states that a field can be extended by the root of a polynomial equation. If the condition defining an infinitesimal could be stated as a polynomial equation the process of constructing an infinitesimal would be easy. The situation is more complex. An infinitesimal ε, as defined for example by Jerome Keisler in his Foundation of Infinitesimal Calculus, is such that for all positive real r, |ε| < r. This includes zero as an infinitesimal, the only real number infinitesimal..

From this definition it follows that if ε and δ are infinitesimals then ε+δ is also an infinitesimal, possibly zero. Consider any real positive r. Then |ε|1 then |ε|Keisler found it convenient to give an equivalent definition (for positive infinitesimals) as

{ε>0, ε<1, ε<1/2, ε<1/3, ε<1/4,..,1/n, … }
for all positive integers.

Note that in Keisler's formulation it is not clear that there would be any way to establish that the product of any two infinitesimals is zero, the condition which was thought to makes the concept useful.

The Extension of the Real Numbers
to the Extended Real Numbers,
the Hyperreals

First consider RN, the set of real-valued sequences; e.g., . The proper notation for such a sequence is < rn: n∈N>. The sum and product of two such sequences are defined in the obvious way:

• < rn: n∈N> + < sn: n∈N> = < rn+sn: n∈N>
  
• < rn: n∈N>*< sn: n∈N> = < rn*sn: n∈N>
• There is an additive identity; i.e., < 0, 0, 0, … >.
• There are additive inverses; i.e., the additive inverse of < rn: n∈N> is < -rn: n∈N>.
• There is a multiplicative identity; i.e., < 1, 1, 1, … >,
• but there is no multiplicative inverse for any sequence that contains even one 0.

The construction of a field involving these real-valued sequences must therefore take a detour.

Consider the set of values of the index n such that the components of two sequences are equal; i.e., {n ∈ N: rn=sn}. One could define the condition of two sequences agreeing almost everywhere if the set where they do not agree is finite. It would then be true that if a sequence <q> agrees with <r> almost everywhere and <r> agrees almost everywhere with <t> then <q> agrees with<t> almost everywhere. This would be the basis for defining an equivalence relation on real-valued sequences, RN. The construction used in nonstandard analysis for creating infinitesimal makes use of a different equivalence relation, but it serves the purpose of explaining the nature of infinitesimals to make use of the above relationship. For more on the alternate equivalence relation click here.

An equivalence relation creates equivalence classes. In the case of the creation of the rational numbers from the integers

{1/2, 2/4, 3/6, 4/8, … }

is an equivalence class that is usually identified with 1/2. In the case of real numbers as infinite sequences of digits one equivalence class is {1.000…, 0.999…} which is usually represented by 1.000…

The set of all real-valued sequences which are equivalent to the sequence is denoted as [q]. The extension of the real numbers is the set of these equivalence classes.

Addition and multiplication can be defined on these equivalence classes.

[q] + [r] = [q+r]
[q]*[r] = [q*r]

These expressions say that the sum of the equivalence class with q as a member and the equivalence class with r as a member is the equivalence class of the sum of q and r. Likewise for the multiplication. It must be established that the result does not depend upon which members of the equivalence classes were selected as representatives. If this were not true then the addition of equivalence classes would not be well-defined.

The existence of additive inverses for the equivalence classes is simple; i.e., the additive inverse of [q] is [−q]. The multiplicative inverse of [q] is constructed from the representative of the class q by generating a sequence r according to the rule

rn = 1/rn if rn≠0
and
rn = 0 if rn=0

Now q*r will not be exactly equal to <1, 1, 1, … >: but it will be equal to it almost everywhere. Thus q*r belongs to the equivalence class of <1, 1, 1, … >:. In other words, [q*r] = [<1, 1, 1, … >:]. Thus, providing that every step is well-defined, [q]*[r] = [q*r] = [<1, 1, 1, … >:], the multiplicative identity of the system. Therefore every equivalence class has a multiplicative inverse.

This means that the system of equivalence classes is a field.

Furthermore an order relation can be defined on these equivalence classes so the field is an ordered field. For two real-valued sequences q and r

• q<r if qn<rn almost everywhere
• q = r if qn=rn almost everywhere
• q>r if qn>rn almost everywhere

It is here that a more sophisticated definition of almost everywhere has to be defined. Accepting on faith that such an order relation can be properly defined then the equivalence classes of the real-valued sequences constitutes a proper extension of the real number system. This is the field of hyperreal numbers, *R.

An infinitesimal in *R is the equivalence class of, for example, the sequence

< 1, 1/2, 1/3, 1/4, … >
Thus ε = [< 1, 1/2, 1/3, 1/4, … >]

This sequence is nowhere equal to < 0, 0, 0, … > so ε≠0. For any sequence of constants, say < k, k, k, … > it can be established that ε < < k, k, k, … >. Thus no matter how small k is, so long as it is positive, ε is less than k. (The set of sequences of the form < k, k, k, … > constitutes the subfield of *R which is isomorphic to the real number system R.

Properties of the Infinitesimals

As was done previously, it can be shown that the sum of two infinitesimal is an infinitesimal, possibly the zero infinitesimal. Likewise the product of any real number and an infinitesimal is an infinitesimal.

The inverse of ε is the equivalence class of

ω = < 1, 2, 3, … >

which is in the nature of an infinite number, but not necessarily having the same properties as an infinity defined in another mathematical system. For the moment call ω an infinitude. In particular, ω < ω2. But the crucial property of infinitudes is that they are multiplicative inverses of infinitesimals. In general, the product of an infinitude and an infinitesimal is a finite real number.

Note that the order of ω relative to that ω² is 1/ω, which is the infinitesimal ε. Likewise it is the case that the order of ε² relative to ε is infinitesimal.

The Structure of the Hyperreals

The field *R has a set of infinitesimals clustered infinitesimally close to zero like a halo. Because *R is closed under addition and contained an image of R, there is a similar halo about each real number. (Because the infinitudes are also elements of *R there is a halo of infinitesimals about each infinitude.)

The elements of *R are partitioned into equivalence classes based on the relation of differing by an infinitesimal; i.e., x and y are in the same equivalence class, denoted x≈y, if x-y is an infinitesimal. The single real number in each equivalence class is the obvious choice as a representative of the class.

The Extension of Functions
Defined on the Reals
to the Hyperreals

The extension of a function defined on the reals to one defined on the hyperreals is much easier than one would expect. A hyperreal is an equivalence class of sequences of real numbers. Consider a function f:R→R. Let x be any hyperreal and let ={rn: n∈N} be any element of the equivalence class x. In other words, x=[]. Then {f(rn): n∈N} is a sequence of real numbers and belongs to an equivalence class in *R, the hyperreals. Then the extension of the function f() to the hyperreals, *f:*R→*R, is simply,

f(x) = []

Let h[x] be the halo of x, the equivalence class of hyperreals which includes x. The continuity of a function f defined on *R can now be expressed as follows:

A function f() is continuous at point x0 if and only if

f(x)≈f(x0) for all x in h[x0].
f(h[x0]) ⊆ h[f(x0)]

For example, consider the function f(x)=x². Then

f(x+ε)=x²+2x*ε + ε²

Since 2x*ε + ε² is an infinitesimal, f(x+ε)≈f(x). Therefore x² is continuous for all real x.

Now consider the function g(x)=2^x. Then g(0)=1. Then g(ε) by the generalized binomial expansion is (1+1)ε=1+ε+b*ε2 for some real b. Therefore g(ε)-g(0) is an infinitesimal and hence g(x)=2x is continuous at x=0.

These same results are of course also easy to prove using the Weierstrass ε-δ procedure but it is notable easier using the concept of infinitesimals. This leads to an aspect of what has come to be called the transfer principle. Since some results are easier to prove in the hyperreals than in the reals it is expedient to transfer the site of the proof to the hyperreals. Since the hyperreals contain the reals if something is true for all hyperreals it is therefore true for all reals.

Note that there is no necessity of ε² being zero; it just has to have zero effect on the equivalence class that a sum belongs to.

The Derivative of a Function
in the Hyperreals

The derivative of a function f(x) at real number x is equal to a real number L if and only if for any nonzero infinitesimal ε

(f(x+ε)-f(x))*ε-1 ≈ L

There is of course no problem of there being a multiplicative inverse for a nonzero ε. Take the simple example of f(x)=x² again.

(f(x+ε)-f(x) = 2x*ε + ε²
and hence
(f(x+ε)-f(x))*ε-1 = 2x + ε
and thus
(f(x+ε)-f(x))*ε-1 ≈ 2x.

If a function f() is known to have a derivative f'(x) at x then the statement

f(x+ε) = f(x) + f'(x)*ε

is a statement about equivalency classes.

The Application of Infinitesimals to Integrations

See Three Approaches to Integration.

(To be continued.)

Filters and Ultrafilters

To define the proper equivalence relation it is necessary to consider the concept of filters and ultrafilters. A filter F is a collection of subsets of a set S such that if A and B belong to S then the intersection of A and B also belongs to F. A filter furthermore requires that if A belongs to F then all supersets of A belong to F. An ultrafilter U is a filter such that for any subset A of S, either A belongs to U or the complement of A in S, Ac=(S-A), belongs to U (but not both).

The equivalence relation on the set of real-valued sequences that is relevant is based upon a non-principal ultrafilter on the real-valued sequences. (Nonprincipal means that the ultrafilter is not generated by only a single element.) Two sequences, and , are equivalent modulo the ultrafilter U iff

{n ∈ N: qn=rn} ∈ U

Sources:

H. Jerome Keisler, Foundations of Infinitesimal Calculus, Prindle, Weber & Schmidt, Inc., Boston, 1976.

Robert Goldblatt, Lectures on the Hyperreals: An Introduction to Nonstandard Analysis , Springer-Verlag, New York, 1998.