At Recordare we have developed a MusicXML format for representing musical scores. The DTDs and more information are available at www.musicxml.org/xml.html. Naturally, we would like to store music in a database and be able to search on it.
The W3C’s XQuery Use Cases and XQuery 1.0 documents are very helpful for learning the basics of XQuery (the use cases in particular). However, I cannot find a simple way to do one of the most fundamental types of music queries. I would appreciate it if people could tell me either 1) what I’ve missed in XQuery or 2) suggestions on how to modify XQuery to better support this type of query.
Say we are trying to search for “Frere Jacques” sung in the key of C. We just want to look for a sequence of pitches: C, D, E, C. We do not want to assume, though, that all these pitches are in the same measure - both of these XML data should be found (this is not real MusicXML, but simplified for this example):
C D E C
as well as
C D
E C
In both cases, we want to return the 4 notes in sequence, regardless of whether they are in the same parent or consecutive parents. However, the following would not match, since there is an F in the middle of the sequence:
C D
F E C
I know there are brute force ways to do this, but this is already a much simpler than real-world query example, and needs to be 1) handled simply and 2) retrieved quickly from an XML database.
Is there some convenient mechanism to specify that I want to match a sequence of elements that share, not necessarily a common parent, but a common grandparent, great-grandparent, or arbitrary level ancestor? Goodness knows there is a lot in XQuery, so please let me know if I have missed the easy way to handle this type of query.
You are looking for a solution, which utilizes some smart efficient primitive constructs of XQuery. I cannot think of such a solution. However you can implement a query as in some functional programming language as (quip version 1.4.1 should be able to execute this):
#########################################
define function head (LIST $es) RETURNS ELEMENT { LET $t := { $es } RETURN $t/node()[1] }
define function tail (LIST $es) RETURNS listOfElement { LET $t := { $es } RETURN $t/node()[2 TO last()] }
define function containsSequence(ListOfElements $isThis, ListOfElemnents $inThis) returns Boolean { if (empty($isThis)) then true() else if (empty($inThis)) then false() else if (startsWith(tail($isThis),tail($inThis))) then true() else containsSequence($isThis,tail($inThis)) }
define function startsWith(ListOfElements $withThis, ListOfElemnents $thisStarts) returns Boolean { if (empty($withThis)) then true() else if (empty($thisStarts)) then false() else if (head($withThis) = head($thisStarts)) then startsWith(tail($withThis),tail($thisStarts)) else false() }
##functions true() and false() were forgotten to be implemented in Quip version 1.4.1 ##this will be fixed in the next version. so long we make this workaround define function true()returns Boolean{empty(())} define function false()returns Boolean{not(true(()))}
I had left out an important part of the problem before. The elements are within elements, and we want to match a sequence that is split between two elements, but not between two elements (common grandparent needed, but different parents OK).
Michael Dyck suggested another approach on the XQuery list, where the query would look something like this (simplified here to a 2-note query):
{FOR $x in document(“frere-jacques.xml”), $note1 IN $x//note[.=“C”], $part IN $note1/ancestor::part, $note2 IN $note1/following::note[1][.=“D”][./ancestor::part == $part] RETURN
{$note1 $note2} }
This looks promising, but I cannot get this to run in QuiP. I get a message “The kind must be one of the constants!” It looks like the problem occurs when using the ancestor:: axis.
Is this XPath axis supported in QuiP? There is a comment in the XQuery working draft that XQuery might not support all XPath axes, and/or may not support both the full and abbreviated syntax. How might we register that it is indeed very important for XQuery to be compatibile with XPath in this respect? Aside from the namespace axis, I am quite sure we are using all the XPath axes in our existing DOM code and would definitely want to use them in our XQuery code.
Thanks again for the help!
Michael Good Recordare
[This message was edited by good on 10 Aug 2001 at 00:34.]
I do not know about the current state for XPath in XQuery and XPath2.0 but as you noticed Quip does not support full XPath, especially not the full syntax. There is no technical reason fo this … so if you realy have a need for some of the missing XPath axis, I can try to add them to Quip. This should not be too much effort.
That would be great if you could do that, Sven. It would really let me start to explore how we could use XML query (and then Tamino) for music information retrieval. Maybe we could demonstrate it at the Music IR conference this October.
I think the abbreviated syntax axes are already supported. The three additional axes that would really help are ancestor::, following::, and following-sibling::. The first two are already used in my sample query. Eventually all the XPath axes would be useful, but this would be a big help in getting started with XML query via QuiP.
Thank you very much for anything you could add in this area!
I just had a very quick go at the axes and ended up with an implementation that can do these axes; but… … well I am in the midst of other things (i.e. a typechecker, updates…). Therefore my current development version of Quip is rather unstable and I cannot predict its behaviour, I made no tests etc…
…well, and there can still be duplicates in the result set…
However, if you want to play around with the current status of Quip as it is, then contact me by private mail at ‘sep @softwareag.com’.
I can send a you current snapshot of the QuiP.jar and quip.exe then.
I had not had much chance to look at this problem for many months. When I finally did, it was right after the April 30 working draft came out.
I’m not sure when the follows, precedes, <<, and >> operators were added, but this sure seems to do the job for us now.
The query that works in QuiP, using real MusicXML syntax (not the simplified version in my earlier note), is:
{let $doc := document(“MusicXML/frere-jacques.xml”) let $notes := $doc//note for $note1 in $notes[string-value(./pitch/step) = “C”], $note2 in $notes[. follows $note1][1], $note3 in $notes[. follows $note2][1], $note4 in $notes[. follows $note3][1] let $meas1 := $note1/… let $part1 := $meas1/… let $part2 := $note2/…/… let $part3 := $note3/…/… let $part4 := $note4/…/… where string-value($note2/pitch/step) = “D” and string-value($note3/pitch/step) = “E” and string-value($note4/pitch/step) = “C” and (string-value($part1/@id) = string-value($part2/@id)) and (string-value($part2/@id) = string-value($part3/@id)) and (string-value($part3/@id) = string-value($part4/@id)) return
Will the “string-value” calls be replaced by “string” in a future version of QuiP? The “string” function is what seems to be used in the current use cases.
This is great news for us! Thanks again for this great tool.