jeudi 2 mai 2019

limit query results involving multiple data objects

I'm creating a webpage that return results from multiple data objects and each object have their own data access implementation. Let's say objectA can return 1,000,000 objects, objectB can return 2,000,000 objects depending what the user is filtering for. I want to limit the search results to 10000 max but the search criteria could come from objectA or objectB or both.

Let's say the user search for a filter where the field only exists in objectA. This would mean I have to filter objectA, then "join" in the code to get the corresponding objectB. What if the 10000 objectA are returned exactly, BUT this could mean 20000 objects B are returned. This would results in 20000 objects in the search results which violate the 10000 limit.

How would you go about implementing this algorithm where only 10000 objects a returned max. The reason this is needed is because we don't want to overload the our application server with too many objects in memory where multiple users can cause memory overload.

In the past, we would just issue a database query that would get us the count of objectA joined to objectB using SQL and this would give us a count. But let's assume we cannot use SQL anymore and can only access the object using data access layer for each object and "join" them together in the code. Is this the wrong approach?

// Need to limit the results of objectAList "joined" to objectBList to 10k to reduce memory consumption List objectAList = objectADataaccess.Query(filters[]); List objectBList = objectADataaccess.Query(filters[]);

List searchResults = objectAList.Join(objectBList, "some matching criteria");




Aucun commentaire:

Enregistrer un commentaire