TCP itself does not authenticate the receiver in any way. If the attacker replies on your TCP SYN-SENT faster than the intended socket for connection, the connection will be established with the attacker instead. That is basically the way some sort of Man-In-The-Middle attacks work. If you remember the Yahoo-highjacking by NSA last year - this worked alike (in that case according to the press even with full TSL Auth.: looking for request to Yahoo, changing some DNS servers to impose to be the receiver (the fake server was physically nearer to the sender and thus faster responding), issue a flasificated TLS Cert making sure the false Cert was verified faster than an original one (as above), and highjacking the connecting therefore.)
Yes, for a MitM attacker it is trivial to respond to a SYN packet with a response that would be considered valid; btw, the ACK number is not even taken from the SYN packet (it is generated by the recipient of the SYN). A much more interesting question is how an off-path attacker is able to guess the correct values and respond (without trying all values, of course)... see papers on off-path TCP injections (there are few, several by Yossi Gilad and me).