The total number of texture sets will have an impact on performance in the viewport. After 15/20 texture sets (if all at 4K resolution) performance can drop quite a lot. We recommend at least to hide texture sets that are not necessary via the texture set list while working. Splitting a mesh in multiple projects can be another solution. A another good idea is to switch the resolution of unused texture sets to a lower resolution. This will help the viewport and avoid loading too many big textures in VRAM. Remember that Substance Painter is non-destructive and the resolution can be changed back to a higher value during the export.