We study by Markov chain analysis the random walks on a critical percolation cluster embedded in a four-dimensional hypercubic lattice. We calculate the number of dominant eigenvalues of the transition probability matrix and estimate the spectral and fractal dimensions d_s and d_w of random walks from the eigenvalues and their distribution. The estimates of d_s and d_w obtained from the data for a given size S of the percolation cluster exhibit some S dependence. Extrapolating the results to S limit, we obtain d_s = 1.330±0.010 close to the previous result by other methods and a new result d_w = 4.50±0.15. These values are also confirmed by direct Monte Carlo simulations of random walks on a percolation cluster.