The IPv4 header is 128 bits (160 with options field) and the IPv6 header i 288 bits. IPv6 headers are easier to parse but nowadays many routers suppport hardware based, wire speed IPv4 and IPv6 routing. On the routing level IPv6 will always add the time to transfer the extra 128-160 bits to each hop in the network compared to IPv4.
The IPv4 header contains information like version, header length, Type of Service, Total packet length, identification number, fragmentation flags, time to live, protocol type, header checksum, source IP address, destination IP address and optional fields if any. The size of the IP address for source and destination is 4 bytes each.
The IPv6 header contain information like version, priority, flow label, payload length, next header, hop limit, source address and destination address. The size of the IP address in IPv6 is 16 bytes each. The IP6 protocol header has been designed to support huge number of IP addresses and fast processing in the Router as the flow label is supposed to help in fast routing of packets and help preserve certain properties of connection and help in creating virtual circuit type of connections. So, as such the routing using IPv6 should be more faster. However, because of increased address size from 4 bytes to 16 bytes one may get some more delay and transmission header length is increased.
I am not actually sure that the assertion of a performance "penalty" in IPv6 is entirely justified, at least not as presented in the above: in some non-trivial cases, an "IPv6 performance bonus" has indeed been observed.
Eric Vyncke and Marcel Eneguehard have tried to make some