When a broadcast audience is selected with the filter, today that audience is fixed. This means if it's set to a tag value, only people with the tag when the filter is created will be included. This should be dynamic, so that the filter is applied at the time the email is sent, not at the time the filter is created. It also appears that people who have unsubscribed will still get these broadcast emails if they are scheduled before the contact actually unsubscribes as they are "on" the list of recipients, even though they have unsubscribed.