It's good to provide only favicon.ico and not favicon.png:
If the favicon.ico not is in the root folder, you should consider adding one, because many tools and services (eg: bookmarking sites, feed readers, web crawlers, ...) request a favicon.ico from the site root, not reading from website html, and it will response a HTTP 404 if favicon.ico not available.
As the time writing this post, only Firefox and Safari support SVG favicons, so we do not need to use SVG favicons now.
For the basic, it only requires favicon.ico for the website, and sometimes favicon.png. But it is better have all of favicon files because there are a lot of browsers and devices with different resolutions. Each files will be used in some specific case to ensure a good displaying for it.