Average mortgage rates drifted slightly higher to end the week, though they remained under the levels seen on Monday and Tuesday. Even then, none of this week's movement was especially abrupt. That's interesting considering there was a decent amount of econom…