Since you already have the ability to fly, the inverted castle is less about collecting the means to proceed, and more about exploration. Your main goal is to find the five pieces of Vlad. To do this you'll have to hunt down five bosses. There's plenty to find in the inverted castle if you take the time to look.
If you want to get started on collecting Vlad's items right away, head towards the Reverse Clock Tower.