I’ve worked on projects that were pretty much green-field with little technical debt but I’ve still had difficulty getting the team to focus on non technical stories.

The problem here was of course that we had a lot of distance between ourselves and our true customer and they didn’t really do the prioritisation. So I.T. were left to do whatever they wanted which was changing the build and cool techie stuff.