Newton-Raphson

To illustrate the iterative procedure of Newton-Raphson as it applies to the multinomial logistic regression model, we need an expression for Eq. 20. Let $\boldsymbol{\mu}$ be a matrix with $N$ rows and $J-1$ columns, the same dimensions as $\boldsymbol{y}$ and $\boldsymbol{\pi}$, with elements equal to $n_i \pi_{ij}$. Then, Eq. 21 expresses a matrix with $K+1$ rows and $J-1$ columns, the same dimensions as $\boldsymbol{\beta}$. By matrix multiplication, the elements of this matrix are equivalent to those derived in Eq. 32.

The expression for the matrix of second partial derivatives is somewhat different from that derived in the binomial case, since the equations in Eq. 37 differ depending on whether or not $j^\prime = j$.

For the diagonal elements of the matrix of second partial derivatives, i.e., where $j^\prime = j$, let $\boldsymbol{W}$ be a square matrix of order $N$, with elements $n_i\pi_{ij}(1-\pi_{ij})$ on the diagonal and zeros everywhere else. Then, Eq. 22 generates a $K+1 \times K+1$ matrix. However, we can only use this formulation for the diagonal elements. For the off-diagonal elements, where $j^\prime \neq j$, define $\boldsymbol{W}$ as a diagonal matrix with elements $n_i\pi_{ij}\pi_{ik}$, and use the negative of the expression in Eq. 22.

Using this dual formulation for $\boldsymbol{W}$, each step of the Newton-Raphson method can proceed as in the binomial logistic regression model, using Eq. 23.