I have a web application with different layers (i.e. ui->services->core). If I have to update one of the layer I have to restart the whole application. So I decided using OSGI to seperate these layers. Then I could update each layer without restarting.
Is this a good idea or is osgi not the right approach?
Well, I could not say if it is the approach, but certainly OSGi is a good approach. As pointed out by kctang in its answer, it has quite a steep learning curve. However, it pays in the end. Your strict modular approach allows you to achieve a separation of concerns that is simply not possible with other frameworks.
Referring to your specific case, I recently read a chapter from OSGi in Depth where they make the point for layer separation in OSGi, with a particular focus on cloud deployment. Summarizing, the Remote Services specification would help you distribute your layers onto multiple machines, which can be a life-saver for enterprise applications.
Another interesting book for you would be Enterprise OSGi in Action, however it is currently available only through the Manning Early Access Program.
If I have to be critical, JMX support by Framework implementors is still missing. Not that you cannot manage your updates in other ways (programmatic or not), but that would have been useful. Anyway, this is just to say that OSGi has room for improvement.