In Windows 10 and Windows Server 2019, it may happen that File Explorer does not display the offline file icons correctly.

And for Windows 11? File Explorer in Windows 11 comes with the same search tools as outlined here, so a similar process can be followed to navigate it.