The last time I was asked to produce my foreigner ID by Japanese police was in 1985 and it ended with the senior officer chastising the junior for calling me "Jones-kun" because I was 20 years old and thus deserved the more polite “-san” suffix.

Thanks to the internet it is easy to confirm that other non-Japanese residents and visitors may have more frequent encounters of this sort, so I realize my experience may not be representative.

Being asked by police to produce the zairyū card or passport foreign nationals are required by law to carry undoubtedly makes you feel singled out for being, or even just looking, foreign.