A group is a set of users that have a common position, goal, or concern. Using groups makes it easy to ensure that you include the right people in new conversations or when posting new documents and such. For example, you probably want to create groups for each department in your organization, but also for each project or committee, or for other designations that fit your company.
The great thing about using groups is that you can make sure no one is accidentally left out of a discussion. And, if someone new needs to be included, you can add them to the group rather than tediously adding them to each conversation, social object, and such that they need to know about.