using stop words

Hello,

In order to perform optimizations to search, I’ve recently loaded a stop words list into ino:stopwords tamino collection. It works perfectly well. When I’m searching for a word which has been added to this list, any result is given.

The point is that when I’m searching an exact matching on a complete expression containing at least one of these stop words into tamino database, I’don’t have result neither.

For example, since I’ve added stop words “du” and “les” into ino:stopwords collection, exact matching on “les amis des animaux” don’t give results anymore.

Is there a way to avoid these kind of problems (custom index ? any tamino function ?)

Arnaud

Hello Arnaud,

you correctly observed Tamino’s current behaviour: when the search term contains a stop word, you will not find a match. The only way to avoid this is to avoid stop words in the search term.
Of course you could also filter the search term by XQuery yourself, e.g.

[code]

declare namespace my=“http://my.ex
declare namespace ino=“http://namespaces.softwareag.com/tamino/response2
declare function my:tokenize($s as xs:string) as xs:string* {
if (substring-after($s," ")) then
(substring-before($s, " “),my:tokenize(substring-after($s,” "))) else $s
}
declare function my:filterSearchString($s as xs:string) as xs:string {
string-join(
let $stop:=collection(“ino:vocabulary”)/ino:stoplist/ino:word/text()
let $new:=for $w in my:tokenize($s) where not($w=$stop) return $w
return $new, " ")
}

{-- sample call --}
my:filterSearchString("es war einmal ein armes M

Thank you for such a fast answer,

I’ve carefully read your solution, wich is based on user defined functions and conditional expressions using ‘if then else’ syntax.
These features are only available on tamino 4.4, aren’t they ?
Unfortunately, our project requires a linux environment (Suse Linux Entrerprise Server 9), which is not supported by Tamino 4.4 yet, I guess.

I have implemented a pretty complex query on tamino 4.2.1 and the syntax (B[A], C)[1] (that replace if(A) then B else C) does not always give expected results (mostly when you mix it up with expressions like : “[position() < …]”).

Best regards,
Arnaud

Hello Arnaud,

my “solution” of course is more a work around, and you are right, it is based on Tamino 4.4. On Linux, this version of Tamino is available since Nov. 2005.

I am interested in teh problem you mention for (B[A], C)[1],
could you please give a full example where it does not work as expected?

Best regards

Harald

Hello,

I don’t have this request anymore. We decided to use tamino4.4 because of a real need of user defined functions. Nevertheless, it seems that errors occurs because tamino4.2 does not really like queries with multiple path in tamino containsText function :

declare namespace tf=“http://namespaces.softwareag.com/tamino/TaminoFunction
(
let $ensemble:=(
for $a in input()/root
where tf:containsText($a/node1 | $a/node2 | $a/node3 | $a/node4, “paris”)
return $a
)

for $b in $ensemble
return

) [position() > 0 and position() < 9]

This does not happen anymore on tamino 4.4

Thank you anyway

Best regards
Arnaud