The ALL section is restricted to 1000 items, nobody's gonna browse more than 20 pages anyway.
So use the letter tabs to dig deeper. Click the little triangle to sort on artist name, number of views or number of videos.
Please Note: the letter is always the first letter of the name of the artist or band. In case of artists that is the first letter of their first name, so Bruce Springsteen will be under B and NOT under S (where he should have been). That's because they are stored that way in our database. The Beatles will be under T and not under B. Just remember that.

Click on a row or on the little round arrow button to go to the artist page.

You can also access the artist page from every video page directly, look for the artist name tag, or from the "Same Artist" tab there as well.