Is the system time invariant?
No, the system is not time invariant because the output of an input signal shifted some value in time does not equal the output of the original signal, shifted the same value in time. In this system the coefficient or amplitude of the shifted output signal changes with time.
What input will yield an output of Y[n]=u[n-1]?
The system seems to work specifically on delta functions, so I take the approach of describing u[n] as an infinite sum of shifted deltas:
$ \ u[n] = \sum_{k=0} \delta [n-k] $
Inputting this function into the system we get the output:
$ \ u[n] = \sum_{k=0} \delta [n-k] \longrightarrow SYS \longrightarrow y[n] = \sum_{k=0} (k+1)^{2} \delta [n-(k+1)] $
So now representing $ \ Y[n] = u[n-1] $ as a sum of deltas we get:
$ \ Y[n]= \sum_{k=0} \delta[n-(k+1)] $
So now there must exist some input of the form $ \ x[n] = B \sum_{k=0} \delta [n-k] $ such that the output is equal to Y[n] above.
Therefore, looking at the output of the unit step and comparing it to the output we want Y[n] we see that if y[n] is multiplied by $ \frac{1}{(k+1)^{2}} $ we get $ Y[n] $. Therefore to get $ Y[n] $, the corresponding input $ x[n] $ must be:
$ \ x[n] = \frac{1}{(k+1)^2} \sum_{k=0} \delta [n-k] $
