That is maybe not the best place.
Go to: Congregations and Speakers (3rd option in the side bar: top is Home, 2nd is Persons, 3rd is Congregations and Speakers)
Select the congregation, then the speaker under the congregation. Click the stylus icon, and untick all talks for that speaker.
Now there are 2 options:
If the speaker is NOT from your own congregation, while still in the speaker's dialogue click the minus (-) sign on the top. It asks if you want to remove the speaker. Click Yes.
(if you do not follow this order, the speaker may be removed, but the connection between the talks, the speaker and that congregation may remain intact in the background)
If the speaker is from your own congregation, maybe it is better to go to the persons table, find the person, and untick 'Public Talks' in the person details.