Washing face only with water cured my oily skin

So since I moved to a new place we didn’t happen to move all my skincare stuff so I began washing my face with just water. for some clarity i have had oily skin for like ever and used many products that were supposed to fix but i still suffered. but after a week i noticed my skin was so much less oily and even when I didn’t put any creams felt moisturized. been a mounth but Only downside is that I would get some minor acne from time to but I don’t wanna go back having my nose dripping with oil any tips to help?