phrase usage - go to university or go to the university

I'm a bit confused. I know that "go to university" is a collocation which means going to attend university. What if I want to say that I'm heading to, walking to, the university. Can I say "I'm going to the university tomorrow morning"?