How can Bob decrypt the message?
Bob can take the encrypted message vector, split it into sub-vectors of three elements each, multiply each sub-vector by the inverse of the encryption matrix, then reassemble the resulting vectors into the original message.
Can Eve decrypt the message without finding the inverse matrix?
Yes, she can. If she knows the size of the encoding matrix and splits her intercepted plaintext and encrypted vectors accordingly, she can write the plaintext as a weighted sum of the encrypted vectors. This is similar to finding the inverse matrix, since matrix operations are, in fact, linear.
She has all the information she needs to actually find the inverse matrix. Once she finds the inverse matrix, she can decrypt any communication she likes provided it is encrypted using the same method.
She could conceivably use Cramer's Rule to decrypt the data one letter/byte at a time, but then she would have to know the original 3x3 encoding matrix.
What is the decrypted message corresponding to [ 2 23 3 ]?
By taking our three sets of inputs and outputs, we can form a system of three equations in three unknowns and solve for the encryption matrix. Taking the inverse gives
$ A=\left[ \begin{array}{ccc} -\frac{2}{3} & 0 & \frac{2}{3} \\ 0 & 1 & 0 \\ 4 & 0 & -1 \end{array} \right] \rightarrow A^{-1}= \left[ \begin{array}{ccc} \ \frac{1}{2} & 0 & \frac{1}{3} \\ 0 & 1 & 0 \\ 2 & 0 & \frac{1}{3} \end{array} \right] $
Multiplying the inverse matrix by the three-element vector given in the homework statement gives
$ \left[ \begin{array}{ccc} \frac{1}{2} & 0 & \frac{1}{3} \\ 0 & 1 & 0 \\ 2 & 0 & \frac{1}{3} \end{array} \right] \left[ \begin{array}{c} 2\\ 23\\ 3\end{array} \right] = \left[ \begin{array}{c} 2\\ 23\\ 5 \end{array} \right] $,
which is "BWE" using our letter-number mapping scheme.
