I have also the experience that D7 is slower than D6, but after some tweaking I think the difference is minimal, I have Drupal running on a shared host, based on comments of other people and my own experience: Drupal is slow Client Side because some modules use javascript and weigh heavily on your own CPU, so:
and Drupal is slow because of Server Side Caching and things
Now my Drupal seems still a bit slow on admin pages (with several modules installed), but not much slower than D6.
Turn on Keep Alive to allow more than one part of a web page to be downloaded from the server at once. Change the settings in the apache config (/etc/httpd/conf/httpd.conf on CentOS)
compressing the CSS file from Drupal can really help pages. Download the CSS_Gzip module and install it.