Now I am puzzled. I ran the same gcode 3 times (the one that was loosing so many steps). Once with sherline off and 2 for pw. Was slightly off, maybe 1 mm. Ran same again with screen off, same result. Ran same again with pw 5 and sherline mode, SAME result.

That must be mechanical, but clearly I got rid of almost all of that mechanical issue because it's so much closer and consistent.