A beautiful red box in Grails documentation says that pagination doesn't work when using listDistinct() method. One should rather use HQL for pagination. Well, I have wrote really huge query using createCriteria().listDistinct() and the citation below caused me almost heart attack.
The listDistinct() method does not work well with the pagination options maxResult and firstResult. If you need distinct results with pagination, we currently recommend that you use HQL. You can find out more information from this blog post.
I could rewrite it to HQL but the query I wrote is really heart of the system. So, it could take me few weeks to rewrite it and I don't have time and energy to do it. I am willing to scarify a bit from performance of the system.
I have discovered the solution described below. It is not the best one from performance point of view, BUT it works and it is not causing high consumption of memory (there are thousands of records in DB nowadays and it works well).
class ResponseService {
def load( def params ) {
projections { distinct ( "id" ) }
// here are your restrictions...
// e.g. eq( 'name', parameter.name )
}
if( idList ) {
// define range startIndex..usedMax
int idListSize = idList.size()
int usedMax = params.startIndex + params.max - 1
def usedIds
if( usedMax >= idListSize ) {
usedMax = idListSize
}
// select only IDs we want to fetch
usedIds = idList[ params.startIndex..usedMax ]
// fetch all Responses based on selected IDs
def results = Response.getAll( usedIds )
return results
}
}
}These resources inspired this solution:
Thanks for this post! Worked perfectly for me...especially converting from using createCriteria().list(max: 10, offset: 1). I started noticing duplicates and was not too thrilled when I read I was going to have to switch to HQL.
ReplyDeleteI do not think it is a good idea to fetch all the ids to idList.
ReplyDeleteAnd I failed to use panigation feature to get idList. totalCount of the PagedResult is wrong, it is the result of "select count(*) from ...", not "select count(distinct(id)) from ...".
Yes, it is not good solution, but it is a solution that helps with performance at some level. Of course, this is stupid solution for the big amount of data.
ReplyDeleteBut, I don't know what exactly is wrong in your case... How did you solve it?
I also come same problem what if we make separate query for fetching results and there total count.
ReplyDeleteSince i also agree that above solution is not very scalable since it will hit perfomance if we have large amount of data.