Link to stockcharts.com (Click on Animate!)

Suppose the probability of default during a year $t$, given that there was no default before year $t$, is $p(t)$. We are considering a loan with term $T$ and constant annual interest $r$, with a principal of 1 and no dividend payments. We stipulate that the expected money we get back at time $T$ should equal the principal. So
$$
1 = \mathbb E((1+r)^T \cdot 1_{\text{no default}}) = (1+r)^T \prod_{t=1}^T (1-p(t))
$$
This gives
$$
\ln(1+r) = \frac{1}{T} \sum_{t=1}^T (-\ln(1-p(t))
$$
We use the approximation $\ln (1+x)\approx x$ and get
$$
r \approx \frac{1}{T} \sum_{t=1}^T p(t).
$$
To use calculus let us instead write

$$
r(T) = \frac{1}{T} \int_{t=1}^T p(t) dt.
$$
Now $dr/dT>0$ if and only if $p(T)> \frac{1}{T}\int_{0}^T p(t)dt$, in other words $p(\cdot)$ is increasing. To use the approximation we must assume that $p(\cdot)$ nevertheless stays close to $0$ in any single year.

So the yield curve being increasing corresponds to an increasing risk of default per year. Now notice that $p(t)$ is bounded above, certainly by 1 and hopefully by something much less than 1.

But whenever $p(t)$ is an increasing and bounded function, then its running average $r(t)$ will be increasing and converging to $\lim_{t\to\infty}p(t)$. Hence $r(t)$ will increase by less and less as we approach $\lim_{t\to\infty}p(t)$.

If the debtor has not defaulted by the year 2200, then it may be because some more favorable conditions may have set in than the present ones. Thus it is reasonable to think that $p(t)$ is decreasing for very large $t$. But a typical scenario where a concave down yield curve is expected is something like U.S. Treasury bonds, which have been solvent for much longer than the term $T$ (say, a couple of centuries compared to at most 30 years). So if the U.S. is solvent in 2042, the conditions are probably no better than they are now. It is more likely that the present stable conditions will have loosened a bit, forcing $p(t)$ to increase.