All the Google penalties can broadly be categorized into two categories: Manual penalties and algorithm penalties. The manual penalties are the penalties imposed on your website for n number of reasons. They are imposed after a manual review of your website by a Google employee. These could be due to a high number of unnatural links, spammed content, hidden text, keyword stuffing, or low-quality content.
Algorithm penalties like Penguin or Panda are usually due to the algorithm updates.