Remove Selective Access for a module

We encountered a situation where we had selective Access enabled for Regions. 

A user had following access:

Region 1 - Write

Region 2 - Write

Region 3 - Read

Region 4- Read

Region 5 - Write.

We had to create a boolean filter on Regions. Since, the user has only read access to Region 3 and 4, he could not set the filter for these 2 regions.

It would have been great, if we could remove selective access just for this filter module.