A socket is a pseudo-file that represents a network connection. Once a socket has been created (identifying the other host and port), writes to that socket are turned into network packets that get sent out, and data received from the network can be read from the socket.
Sockets are similar to pipes. Both look like files to the programs using them. Both facilitate interprocess communication. Pipes communicate with a local program; sockets communicate with a remote program. Sockets also offer, as you mention, bidirectional communication (much like a pair of properly connected pipes could).