Painting on layers that are below the other layers in the layer stack can have a major impact on performance, similar to Substance Designer when you tweak a node at the source of a graph: by changing a layer early, you end-up computing the layers on top simply because of the hierarchy. A solution to that is to hide the layers on top of the one you are currently working on (temporarily). Then unhide them once you have finished, this way the computation of the layers above will be done only once and not during each brush stroke.
You can also group all the layers above in one individual group with the blending mode set to Normal. Doing so will avoid computing the layers/group individually for the engine. While the Substance Painter engine employs multiple algorithms to determine what needs to be computed or not, grouping manually layers can help in the end.