Recently, I've noticed that in ubuntu (6.06) dapper server with apache-2.0.55, apache by default listens to IPv6, thus was causing slow response times. The response times was much improved by having apache listen to IPv4 instead.
Edit /etc/apache2/ports.conf and specify an IPv4 address on all Listen directives:
Listen 0.0.0.0:80
Listen 0.0.0.0:443
Root cause?
How does IPv6 impact Apache's performance? Do you have v6-enabled clients? Are they using higher-latency v6 routes?
ipv6 network
Not all Internet Service Providers are able to handle IPv6 properly which causes very slow DNS lookups because of timeouts or failure to perform the DNS lookup at all. Most networks currently use ipv4 as default, so the ipv6 sites that are already in use are "translated" into a ipv4 compatible form. This adds to the slowdown.
If you're concerned about
If you're concerned about DNS issues, then remove the AAAA record for your web site. You shouldn't have to adjust Apache's configuration.