and Drupal is slow because of Server Side Caching and things
Now my Drupal seems still a bit slow on admin pages (with several modules installed), but not much slower than D6.
Turn on Keep Alive to allow more than one part of a web page to be downloaded from the server at once. Change the settings in the apache config (/etc/httpd/conf/httpd.conf on CentOS)
compressing the CSS file from Drupal can really help pages. Download the CSS_Gzip module and install it.