The selvage edge of fabric is what attaches to the loom during manufacturing; so it only stands to reason this portion of the fabric needs to be sturdy and firmly woven.
Sometimes the result is a selvage edge that “draws up” and won’t lie flat and smooth when you are trying to cut out a pattern. This tends to happen more with knit fabrics, but it can happen with wovens, too. Next time, rather than cutting the entire selvage off, try clipping into the selvages every few inches to release the tension. Hope that helps!
Happy sewing!
Maris