Authentication is a tough problem. For a user to authenticate himself, he generally has to prove he knows some previously agreed-upon secret without actually transmitting the secret itself. When 802.11 was first designed, the standard specified two types of authentication, open and shared key.
In a strange twist of computer security irony, it turns out that the "more secure" shared secret authentication technique actually made things worse! By enabling shared key authentication, you put the attacker is a position to develop what cryptographers call known-plaintext attacks.
Shared key authentication is the IEEE's first attempt at providing some sort of authentication in 802.11. Before 802.11i, this was the only technique specified in the standard. In 802.11, shared key authentication looks a lot like many basic security challenge-response protocols.
When using shared authentication, the AP sends the client a challenge packet. The challenge packet contains a string of random bytes. The client must encrypt this packet with the secret key and send it back to the AP before she associates. The AP itself also encrypts the challenge packet. If the client sends a correctly encrypted packet back to the AP, she knows the key.
The major design problem with this type of authentication is that anyone observing this exchange can see the plaintext challenge packet and the encrypted response. This allows an attacker to compute the output generated by RC4, giving rise to some advanced attacks covered later in the chapter.
Open authentication is basically "no authentication." Just because an AP uses open authentication, however, doesn't mean that you immediately get a free ride on the network. There are many other places where you can be cut off.
Most networks that deploy WEP nowadays actually use open authentication. Even though authentication is essentially turned off, only users who know the key can transmit packets. If an attacker doesn't know the key, he won't be able to encrypt any packets successfully. Because he won't be able to encrypt them correctly, the AP will just drop the packets. That's the idea anyway. You'll see how to actually encrypt packets without knowing the key later.
When the IEEE was creating 802.11i, they realized that the authentication problem had been tackled previously in another standard. This standard is awkwardly named 802.1X. It is important to note that 802.1X does not specify a specific authentication algorithm; instead, it specifies a framework that generic authentication protocols can be built around. 802.1X authentication is only as secure as the specific authentication technique it is using. An overview of 802.1X is provided in Chapter 3, in the "WPA/802.11i Background" section. Details of a few specific 802.1X authentication techniques are covered in Chapter 8 in the "Deploying WPA/WPA2 Securely" section.
The biggest conceptual difference between 802.1X and the original 802.11 shared key authentication is the following. In the original 802.11 standard, the committee was trying to authenticate a user before she was allowed to transmit any data at all. It turns out this is a sort of chicken and egg problem. To provide secure authentication methods, the client must be able to send certain types of data packets. By precluding the station from sending any data packets at all, 802.11 precluded the use of advanced authentication techniques. 802.1X recognizes this and will allow the user to send data packets as long as they are strictly related to authentication only.