As mentioned before, there is good unity and there is bad
unity. Simply being unified is not a good thing when that unity is done
in an effort to do mischief. A group plotting to rob a bank, for
instance, is certainly not unified in a good way. God had told them to
fill the earth and enjoy it's fruits. They had.the whole new world to
explore. Instead of setting their hearts on honoring God with their
actions, though, they began to let mischief creep in.
Everyone on earth had the same language at this point.
However, they were starting to drift away from the Lord quite a bit.
Noah and Sam were still around, we presume that Ham and Japheth were as
well, and yet people refuse to listen to these wise men. Wisdom is only
helpful if it is listened to and if it is Godly wisdom. Whatever we take
into our minds must be tested against the absolute truth of God's Word.
Being of one language, one wonders if everyone's idioms and
usage of it was the same. I don't think so. There are many differences
between people in the same neighborhood in a small city. However, the
important thing is that they can still understand each other. Context
can be grasped quite easily as long as someone is willing to listen.
And, this was made easier here because - like in that small town - there
is still a major common experience all had in growing up in the same
area and listening to the ancient wisdom from people like Noah and Shem.