Time Invariance
The system is not time invariant. This is plainly seen from the coefficient of the $ \delta $ functional in the $ Y_k[n] $ column. As the input changes, the output changes as the square of the subscript in the input sequence.
Linearity
Since the system is linear, the required input will be $ X[n]=u[n] $. An input of a $ \delta $ functional produces an output of a time-shifted $ \delta $ functional, so an input of a unit step function will produce an output of a time-shifted unit-step function.