Why does Hibernate always initialize a collection when I only want to add or remove an element?

Unfortunately the collections API defines method return values that may only be computed by hitting the database. There are three exceptions to this: Hibernate can add to a <bag>, <idbag> or <list> declared with inverse="true" without initializing the collection; the return value must always be true.

If you want to avoid extra database traffic (ie. in performance critical code), refactor your model to use only many-to-one associations. This is almost always possible. Then use queries in place of collection access.