Associate point data within specified attribute group

I am trying to take a point feature class and find the nearest neighbor from another point feature class but keep it within a certain group. I thought the group by feature would be all I required inside the neighborfinder but when I use this field, my inputs all come out the "unmatched base" port.

 

 

My inputs are zipcodes as point feature class (base) and cities as points (candidate). I want to associate each city to a zipcode. This works fine. However, I am getting a few that are being associated to the wrong state. So, I want to make sure that it finds the nearest neighbor but stays within the state. I have a state_abbr field in the zipcode data. But, like I said, when I set this field to the group by and run the workspace, everything comes out unmatched.

 

 

Is there a way to do this association without the neighbor finder?

 

 

Thanks