Logit

The logistic distribution $$\Psi(x)=\frac{1}{1+\exp(-x)}$$ increases from zero to one on the real line. As we have seen before, the function $$f(x)=x-\log\Psi(x)=\log(1+\exp(x))$$ is strictly convex. This follows directly from $$\begin{align*} f'(x)&=\Psi(x),\\ f''(x)&=\Psi(x)(1-\Psi(x)), \end{align*}$$ because the first derivative is increasing and the second derivative is positive.

Theorem: The r-th derivative $f^{(r)}(x)$ is a polynomial in $\Psi(x)$ of degree $r$. Consequently for all $r$ there are two finite real numbers $m_{r}<M_{r}$ such that $m_{r}\leq f^{(r)}(x)\leq M_{r}$ for all $x$.

Proof: We know that $f'(x)=\Psi(x)$, and thus the result is true for $r=1$. Now proceed by induction. If $f^{(r)}(x)=P_{r}(\Psi(x))$ for some polynomial $P_{r}$ of degree $r$, then $$f^{(r+1)}(x)=P_{r}'(\Psi(x))\Psi(x)(1-\Psi(x)),$$ which is indeed a polynomial in $\Psi(x)$ of degree $r+1$. In addition $$\begin{align*} \sup_{x}f^{(r)}(x)&=\max_{0\leq s\leq 1}P_{r}(s),\\ \inf_{x}f^{(r)}(x)&=\min_{0\leq s\leq 1}P_{r}(s), \end{align*}$$ and the quantities on the right-hand side are clearly finite. QED

We illustrate the theorem by computing some higher derivatives $$f^{(3)}(x)=\Psi(x)(1-\Psi(x))(1-2\Psi(x)),$$ $$f^{(4)}(x)=\Psi(x)(1-\Psi(x))(1-6\Psi(x)+6\Psi^{2}(x)),$$ $$f^{(5)}(x)=\Psi(x)(1-\Psi(x))(1-2\Psi(x))(1-12\Psi(x)+12\Psi^{2}(x))$$ which implies $$\begin{align*} -\frac{1}{18}\sqrt{3}&\leq f^{(3)}(x)\leq+\frac{1}{18}\sqrt{3},\\ -\frac18&\leq f^{(4)}(x)\leq\frac{1}{24} \end{align*}$$ The derivatives of orders 2 to 5 are in Figure 1.

We can go further than this and derive an explicit formula for the polynomials. The difference/differential equation we have to solve is $P_{r+1}(x)=x(1-x)P_{r}'(x),$ where $P_1(x)=1-x$. Its general solution is $$P_r(x)=\sum_{j=1}^r(-1)^{j-1}(j-1)!S(j,r)x^j$$ where the $S(j,r)$ are the Stirling numbers of the second kind (the number of ways of partitioning $r$ elements into $j$ non-empty subsets).

The code in logitPom.R computes the polynomials $P_r$. With logitPomRecursive(n) we compute all polynomials up to order n, their roots, and their maximum and minimum values. With logitPomDirect(n) we do the same, using the formula with Stirling numbers.

$$-\log(1-\Psi(x))=-\log(1-\frac{1}{1+\exp(-x)})=x+\log(1+\exp(-x))$$

$$x+\log(1+\exp(-x))=\sum_{s=1}^\infty\frac{(\Psi(x))^s}{s}$$