How to make a rigorous epsilon-delta proof.

A core skill that you acquire in an introductory real analysis course is how to prove that a function has a limit. Unfortunately this can also be a very common point of confusion for new students. In this post I want to give some guidance and templates for how to think about these proofs.

Definition

This may seem obvious in retrospect, but it is utterly essential that you recall the definition of a limit. POP QUIZ! What is it?

If you cannot recite it right now, verbatim, effortlessly, then this is something you should exercise. Write it down somewhere, take a picture, or whatever, and frequently write this definition over and over throughout the day.

There are a few different versions of the definition, some of them more “analytic” in nature, some of them more “topological”. I’m going to give the simplest one that I can, which is still rigorous.

The function \(f:\Bbb R\to \Bbb R\) has a limit \(L\) at the point \(a\in \Bbb R\) if:

For every positive real number \(\varepsilon > 0\), there exists a positive real number \(\delta>0\) such that

\(0<|x-a|<\delta\) implies \(|f(x)-L|<\varepsilon\).

In this case we write \(\displaystyle \lim_{x\to a}f(x) = L\).

It is very important to also think about the geometrical, visual intuitions for this definition. However, I’m going to trust that someone else has already drawn the pictures for you, and so I won’t do that here. If you’d like to see a visualization, feel free to leave a comment and I can make another post.

It is also valuable to write the above definition in “logish”. Logish is a mixture of “logic” and “English” the way that Spanglish is a mixture of “Spanish” and “English”. Below is the logish definition of the limit.

\(\displaystyle \lim_{x\to a} f(x) = L\) if

$$\forall \varepsilon>0,\exists \delta >0,$$

$$ |x-a|<\delta \Rightarrow |f(x)-L|<\varepsilon $$

Logish is pleasantly compact even if it is notation-dense. Choose whichever definition you like best.

Simple Examples

With the definition in hand, let’s talk about how to prove that some example fulfills the definition.

Before actually talking about broad generalizations about how to prove these claims in a range of settings, let’s look at some simple examples. We will prove \(\lim_{x\to 1} x = 1\) and \(\lim _ {x\to 2} x^2 = 4\) and \(\lim _ {x\to 3} 1/x = 1/3\).

\(x\)

We prove that \(\lim _ {x\to 1} x = 1\), without explanation—just the finished, direct proof. We will comment and reflect on the proof later.

Proof: Let \(\varepsilon >0\) and set \(\delta = \varepsilon\). Then \(\delta > 0\).

If \(|x-1| < \delta\) then \(|f(x) - 1| = |x-1| < \delta = \varepsilon\).

This demonstrates that the definition of \(\lim_{x\to 1} x = 1\) is satisfied.

\(\Box\)

\(x^2\)

$$\lim _ {x\to 2} x^2 = 4$$

Proof: Let \(\varepsilon > 0\) and set \(\delta = \min\{1, \varepsilon/4\}\). Then \( 0 < \delta\) and \(\delta \le 1\) and \(\delta \le \varepsilon /4\).

Now suppose \(0<|x-2|<\delta\). Note that, from algebra, we have

$$ \begin{aligned} |f(x) - 4 | &= |x^2 - 4| \\&= |(x-2)(x+2)| \\&= |x-2||x+2| \end{aligned} $$

Note that we have \(|x-2| < \delta \le \varepsilon/4\).

Also we have \(|x-1|<\delta\le 1\) and therefore \(|x-1| < 1\). In turn, this implies \(-1 < x-1 < 1\) and therefore \(2 < x + 2 < 4\). In particular, this shows that \(|x+2| < 4\).

Putting the lines above together, we may infer

$$ \begin{aligned} |f(x)-4| &= |x-2||x+2| \\ &< (\varepsilon/4)(4) \\ &=\varepsilon \end{aligned} $$

This demonstrates that the definition of \(\lim_{x\to 2} x^2 = 4\) is satisfied.

\(\Box\)

\( 1/x\)

$$\lim _ {x\to 3} \frac 1 x $$

Proof: Let \(\varepsilon > 0\) and set \(\delta = \min\{1, 6\varepsilon\}\).

$$ \begin{aligned} |f(x)-1/3| &= \left| \frac 1 x-\frac 1 3 \right| \\ &= \left| \frac{3-x}{3x} \right| \\ &= |x-3|\cdot \frac{1}{3|x|} \end{aligned} $$

Note that \(|x-3| < \delta \le 6\varepsilon\).

Also note that because \(|x-3|<\delta \le 1\) then \(-1 < x-3 < 1\) and therefore \(2 < x < 4\). In particular this shows that \(2 < |x|\) and therefore \(\frac{1}{3|x|} < \frac 1 6\).

Putting the lines above together, we may infer

$$ \begin{aligned} |f(x) - 1/3| &= |x-3|\cdot \frac {1}{3|x|} \\ &< (6\varepsilon) \left(\frac 1 6\right) \\ &= \varepsilon \end{aligned} $$

This shows that the definition \(\lim _ {x\to 3} \frac 1 x = \frac 1 3\) is satisfied.

\(\Box\)

Reflecting on Strategies

Abstracting from these three proofs, how do you prove that a limit statement is true, in general?

There is no recipe

I tried to present three relatively simple proofs and to present them in a relatively uniform way. But keep in mind that these patterns are not “fool proof”. There are proofs of certain limit statements which cannot be written so simply.

There is no algorithm for proofs. There simply is no sequence of steps to follow, the way that you may be used to in other math classes.

When computing the derivative, there is an algorithm to follow: If the function is a sum, distribute the derivative to each term and then take the derivatives of each. If it’s a product, follow the product rule, and so on. Eventually you break it down until you reach a function with known derivative and apply that rule.

But in mature, advanced mathematics, like in real analysis, things are just too abstract and too general for there to be any reasonable algorithm like that.

However, for a new student trying to find a place to get a toe-hold on the subject, this is a good starting point, and we can give some advice which helps you to do the simpler proofs.

How To Solve

• Always start your proof with “Let \(\varepsilon > 0\).” This is simply how the logic of any proof must proceed—never skip it!

  Moreover, as a general proof-solving technique, you should always try to get things on paper. What do I mean by that? Well if you have a proof to solve, and you’re sitting there staring at a blank piece of paper, then you definitely haven’t done as much as you could be doing. It seems like a trivial step, but by writing “Let \(\varepsilon > 0\).” now puts the number \(\varepsilon\) on your paper, and that gives you something to work with.
  
  I have very often tutored students who give up on trying to solve a problem before they even do this step. That’s a big mistake that’s easy to fix. Once \(\varepsilon\) is on your paper, you can now talk about it, reason about it, and think about what you need next. You can experiment with it, try things out! Maybe it doesn’t work but at least you’re doing something. Maybe with enough effort, false-starts, and mistakes, you’ll eventually figure out the correct proof because at least you’re doing something.

• After “Let \(\varepsilon > 0\).” the very next thing you need to write on your paper is “Set \(\delta = …\)” where the value that you put in the “…” is whatever the rest of the proof requires.

  • Do not skip this step! The logic of a limit statement demands that you specify the value of \(\delta\) which corresponds to the value \(\varepsilon\). Too many students make the mistake of never actually saying what \(\delta\) is, and lose points for it. The entire mission of your proof after letting \(\varepsilon > 0\) is to find the value of \(\delta\)!

  • Ok but how do you find \(\delta\)? That seems so open-ended! Where do you … find it?
    
    For this you need scratch work. My proofs that I showed you up above just have a value of \(\delta\) that kind of “pops up out of nowhere”. In the first proof it was just \(\varepsilon\), in the second it was \(\min\{1,\varepsilon/6\}\), and in the third it was \(\min\{1,6\varepsilon\}\). How did I find those values to know that they would work out in the rest of the proof?
    
    I did scratch work that I did not show you in the proof. This feels a little dirty, right? Hiding your own logic? Well, this is how mathematics is written professionally. We tend to show valid proofs without explaining the thought process which produced the proof. If you look in any math journal or research publication, you’ll likely see this in every single article. Why do we do this? I think it’s for efficiency: When giving the proof, we merely show you the technically correct valid argument. If we tried to express our psychology and process in every single proof, it would multiply the lengths of proofs by at least 2 or 3. Moreover, once you get the hang of it, you no longer need to see people’s thought processes: You can read other people’s proofs and sort of figure out how they did it with a high degree of confidence.
    
    Anyway, regardless of the reasons, you should know that some of the work that you do to come up with a proof is not supposed to be shown in the actual proof—it’s supposed to be “scratch work” written somewhere else.

  • Ok, in the proofs that I gave above, what was the scratch work?

    • For \(\lim_{x\to 1}x =1\) I know that in the end I need to arrive at \(|f(x)-1| < \varepsilon\) which is the same as \(|x-1| < \varepsilon\).
      
      But notice that \(|x-1|\) is precisely the thing that I’m able to “control” by my choice of \(\delta\)! Since I get to (need to) pick the value of \(\delta\), then here it is clear that I want to pick it to be \(\varepsilon\) since this will guarantee that \(|f(x)-1| < \varepsilon\).

    • For \(\lim_{x\to 2}x^2 = 4\) I know that I need \(|f(x)-4| < \varepsilon\) which is the same as (by repeating the algebra that I showed earlier) \(|x-2||x+2| < \varepsilon\). Here it’s less obvious how we pick \(\delta\).
      
      Although part of it is easy. One factor is \(|x-2|\) which is exactly the thing we get to control with the choice of \(\delta\). So we regard this as not really the “hard part”. Let’s focus our attention on \(|x+2|\) first.
      
      We need \(|x+2|\) to be small by an appropriate choice of \(\delta\). How small? Honestly it kind of doesn’t matter. We just need some kind of upper-bound on the quantity. Once we have an upper bound, we can then just pick \(|x-2|\) to be so small relative to that bound, that everything comes out less than \(\varepsilon\).
      
      Ok, so we want to limit how big \(|x+2|\) is allowed to be. The only control we have over anything is through the choice of \(\delta\). Let’s just arbitrarily say that \(\delta = 1\) and see what happens. (This is a part of proving things! Try out a reasonable guess, see how it goes, possibly it fails. If it fails, that’s not always horrible because sometimes in the process you can see why it failed. And then you can come up with a way to fix the failure. But importantly: Just keep going, keep your pencil moving, keep trying stuff out. It’s better than sitting there stuck.)
      
      So if \(\delta = 1\) then \(|x-2| < \delta = 1\). What does this imply for \(|x+2|\)? It’s good to be handy with the fact that \(|x-2|< 1\) implies \(-1 < x-2 < 1\), we will use that kind of thing very often in real analysis. From here we get \(1 < x+2 < 6\). (I added 5 precisely so that I would have an expression containing \(x+2\).)
      
      Ah-ha! I have an upper bound on \(|x+2|\) exactly as needed! The upper bound is 6. I don’t actually care what the bound is, so long as I have some specific number. (It’s ok if the number has \(\varepsilon\) in it, just so long as it doesn’t have x in it. In our case, 6 works because it doesn’t have x in it.)
      
      Ok, so \(\delta = 1\) is nice, it implies \(|x+2|<6\). Now if we can just make the other factor, \(|x-2| < \varepsilon/6\) then the 6’s will cancel and we’ll be left with \(< \varepsilon\) exactly as needed. Again, then only thing you get to control in order to accomplish these goals, is to set the value of \(\delta\) in order to control the value of \(|x-2|\). We therefore need \(\delta \le \varepsilon/6\).
      
      At this point we have two conditions that we need to satisfy simultaneously. We need \(\delta \le 1\) and we need \(\delta \le \varepsilon / 6\). (We previously discussed \(\delta = 1\) but everything that we did with that would come out the same if we only have \(\delta\le 1\). In general, never be afraid to pick a smaller \(\delta\). Choosing \(\delta\) smaller can only give you a better chance of a successful proof, it never harms anything.) In order to accomplish both of these conditions, we can merely set \(\delta = \min\{1,\varepsilon/6\}\). Whatever the minimum is, it is guaranteed to be less-than-or-equal-to each of those quantities.

    • For \(\lim_{x\to 3\} 1/x = 1/3\) the logic is very similar, there is scratch work that I don’t show you. That scratch work is \(|f(x)-1/3| = |x-3|\cdot\frac 1 {3|x|}\). Again, the factor \(|x-3|\) is easy to control, so we don’t really worry about this one right now. Instead we worry about \(\frac 1 {3|x|}\) first.
      
      We need this to be small, and we control it by setting \(\delta\) which controls \(|x-3|\). Again let’s just arbitrarily set \(\delta = 1\) and see what happens. Then \(|x-3|<1\) and therefore \(-1 < x-3 < 1 \). Again I add the amount needed to get the thing I’m interested in, which right now is \(|x|\). So I add 3 to get \(2 < x < 4\). Since I want $\frac{1}{3|x|}$ to be small, then I want \(|x|\) to be big. Therefore I’m interested in the fact that \(2 < x\) since it guaratnees that \(\frac 1 {3|x|} < \frac 1 6\).
      
      Now that we know \(\delta \le 1\) gurantees \(\frac 1 {3|x|} < \frac 1 6\), then we want \(|x-3| < 6\varepsilon\) so that the 6’s will cancel. We can accomplish that directly by ensuring that \(\delta \le 6\varepsilon\). Again, we are trying to make both conditions hold at the same time, so we take the minimum of the two quantities. These are all relatively common patterns when proving limit statements.

Practice!

You do not understand until you practice.

Do not trust your feeling of understanding — your feelings lie!

Believe nothing about yourself except results!

Exercise set 1

Prove each of the following limit statements.

• \(\lim_{x\to -1} (2x+1) = -1\)

• \(\lim_{x\to -2}(-3x^2) = -12\)

• \(\lim_{x\to 0}(x^2+x+1) = 1\)

• \(\lim_{x\to 1}\left(\frac {x^2 + 1}{x+1}\right) = 1\)

These are each relatively simple problems, only a notch more difficult than the examples given above. Keep increasing the difficulty of the limits that you can prove.

Exercise Set 2

It is possible (but a challenge) to prove that for any \(n\in\Bbb N\) and for any \(a\in\Bbb R\) we have that \(\lim_{x\to a} x^n = a^n\).

Hint: You will need to factor \(x^n-a^n\). There is a known formula for this. You can look it up, but you’ll be a stronger mathematician if you can derive it yourself. As a hint for deriving this factorization formula, first let’s just see how this works on a small but not trivial example. Maybe \(n=4\) is good. Find a clever and well-organized way to multiply and simplify the following product.

$$ (x-a)(x^3 + x^2a + xa^2 + a^3) $$

Once you’ve figured this out and see how it gives you a factorization of \(x^4-a^4\), then generalize it.

Exercise Set 3

Notice that the techniques that I gave in this blog post are not sufficient for the following limit proofs:

• \(\lim_{x\to 0}x\sin x = 0\)

• \(\lim_{x\to 0}\frac{\sin x}x = 1\)

• \(\lim_{x\to 0}\frac{e^x-1}x =1\)

Don’t break your back trying to prove these limits, but just work them out enough to see where you get stuck.

The fact that you get stuck might mean you need to just think of somthing clever. Or it might mean that you just shouldn’t even try to approach the problem directly with an \(\varepsilon-\delta\) proof. You might need to instead reach for some relevant theorem. The first one above is best approached with the “squeeze theorem”, and you shouldn’t bother with the direct approach.

In general, you’ll never know for sure which is the right approach. But there are hints and guides:

• If your reading assignment or lecture covered a theorem, there’s a good chance some problem on the homework will need it. Think about the theorems, and which ones might help with which problems.

• In general, try to approach a problem directly. If it gets hard, read all the theorems that you’ve covered, and wonder if it could perhaps apply and save you some work.

• Sometimes theorems don’t apply directly and immediately, but you can rearrange things and apply the theorem in a creative way. If a theorem seems like a nearly good fit, then try to identify the exact reasons why the theorem doesn’t apply—and then wonder, could I consider some related object for which the theorem does apply?

This “exercise set” is not really so much an exercise set as a demonstration. The point here is to be aware of situations when you may not want to approach a problem directly. Don’t bother actually trying to solve the above limit proofs.