I understand your dissatisfaction.
I am not especially happy about it, but I think the current implementation is a good solution, though it is a trade-off.
In the light that most people use dbms to have powerful retrieval functions (and that fast), this trade-off is (as I believe) inevitable.
Imagine the extra stuff that would need to go into the database code, if it handled entities in a special manner. Think of querying on content that is in entities. Storing the document with entity references unresolved would require the kernel to resolve the entities of all documents on each and every query run. Same for indexes.
I think, SAG development had some quite heated discussions on how to exactly handle entities. For the time being I think it’s a good enough implementation, for it is in the name of simplicity and, thus, speed.
I understand that this might not exactly help you. I hope you can come up with another solution that doesn’t require entity references in the query result set. If you could share the problem you’re trying to solve, maybe someone around here picks it up. I think, you’re not the first one dealing with this.
In regard to external entities: Unparsed entites really have no business in the XML store, they are not part of the XML, the concept is merely for linking. The way out is to store the unparsed stuff in a nonXML doctype and employ another mechanism for linking.
Best regards, Andreas