Hover over Contacts and click on My Contacts. You can also click + New Contact for a quick way to add a new person.

Find the respective contact in the middle column and click to select them

On the right side of the page, click Edit

Scroll down to the Group Assignment section. Checking groups will add the contact to the respective group and unchecking will do the opposite and remove the contact.

Click Save when finished