Removing duplicates in doclist

, ,
1

someone please provide removing duplicates in doclist

i dont have java service for removing duplicates in my designer

2

You need to provide more info,
i.e. what dictates a duplicate, a keyword in a complex type, a simple string, the name of a child document etc. etc.

4

Hi Akhil,

would be interesting to learn from where this data is originally coming from and how this is aggregated into the displayed doc.
Most likely there is an issue in the code before the doc is finally populated.

Regards,
Holger

5

Are you saying that the duplicates shouldn’t be there ?
or asking how you could write code to remove duplicates that you expect to be there ?

That’s two different things, the former as @Holger_von_Thomsen has indicated; you will need to backtrack to identify the data that is coming into your service and also perhaps include a screenshot of the code.

If it’s code example to clean up duplicates that you are after, then you can do this most efficiently with a java service. In which case I could post you an example here.

regards,
John.

6

if there is any java service for removing duplicates records in document list … pls provide

7

Sorry for the delay getting back to you.

Here is a copy of a java service that I have in my own tools package

/** 
	 * The primary method for the Java service
	 *
	 * @param pipeline
	 *            The IData pipeline
	 * @throws ServiceException
	 */
	public static final void removeDuplicates(IData pipeline) throws ServiceException {
		// pipeline in
		
		IDataCursor pipelineCursor = pipeline.getCursor();
		IData[] docIn = IDataUtil.getIDataArray(pipelineCursor, "docListIn");
		String searchKey = IDataUtil.getString(pipelineCursor, "key");
		
		// process
					
		List<Object> docOut = null;
		
		if (docIn != null) {					
			ArrayList<String> processedKeys = new ArrayList<String>();
			docOut = new ArrayList<Object>();
			
			for (IData doc : docIn) {
				
				IData notDupedDoc = checkForDuplicates(doc, searchKey, processedKeys);
				
				if (notDupedDoc != null)
					docOut.add(notDupedDoc);
			}
		}
				
		// pipeline out
			
		if (docOut != null)
			IDataUtil.put(pipelineCursor, "docListOut", docOut.toArray(new IData[docOut.size()]));
		
		pipelineCursor.destroy();
	}
	
	// --- <<IS-BEGIN-SHARED-SOURCE-AREA>> ---
	
	
	private static IData checkForDuplicates(IData doc, String searchKey, ArrayList<String> processedKeys) {
	
		IDataCursor docCursor = doc.getCursor();
		docCursor.home();
		
		while(docCursor.hasMoreData()) {
			docCursor.next();
		
			String key = docCursor.getKey();
			Object value = docCursor.getValue();
			
			if (searchKey != null && value instanceof String) {
								
				if (key.equals(searchKey) && !processedKeys.contains(value)) {
										
					processedKeys.add((String) value);
					return doc;
				}
			} else if (value instanceof IData) {
				if (checkForDuplicates((IData) value, searchKey, processedKeys) != null) {
					return doc;
				}
			}
		}
		
		docCursor.destroy();
		
		return null;
	}

You can also download my package from here

service is

jc.tools.pub.idata:removeDuplicates

regards,
John.

1 Like
8

thank you so much…

9

could u plz provide what are the inputs and outputs for this code. Because I’m unable to run this code properly

10

From the code:

/* Input */
IData[] docIn = IDataUtil.getIDataArray(pipelineCursor, "docListIn");
String searchKey = IDataUtil.getString(pipelineCursor, "key");

/* Output */
IDataUtil.put(pipelineCursor, "docListOut", docOut.toArray(new IData[docOut.size()]));

Input
Document List/Array: docListIn
String: key

Output
Document List/Array: docListOut

1 Like
11

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.