You have to press the "jump" button to ascend those kelp-looking scroll thingeys. But it is possible to beat the game without using any mana from the wanderer's scarf.
Edit: Well, I guess that's true also if you use the costume with regenerating mana. Unless you mean not jumping AT ALL, not just refraining from using scroll mana.
One cool thing I found out is that it's possible to ascend the tower without activating any of the shrines. It's a fun little challenge and I think it would have made for an awesome trophy.
I've done it a few times by myself with the white robe. It may be possible with the red robe if you coordinate with another person by refilling each others scarfs with chirps. But it's something I haven't been able to test out because of the whole "being paired up with a random stranger" aspect.
60°
40°
100°
50°
Ad 
80°
10°
10°
20°
40°
40°
btw; i'm a cheater :p
At one point the entire an level you have to go "up".
I'm the best at whatever it is i happen to be doing.
Edit: Well, I guess that's true also if you use the costume with regenerating mana. Unless you mean not jumping AT ALL, not just refraining from using scroll mana.
I've done it a few times by myself with the white robe. It may be possible with the red robe if you coordinate with another person by refilling each others scarfs with chirps. But it's something I haven't been able to test out because of the whole "being paired up with a random stranger" aspect.