How to Paginate Queries by Combining Query Cursors Using Firestorerecycleradapter

Is there a way to paginate queries by combining query cursors using FirestoreRecyclerAdapter?

As @FrankvanPuffelen already answered in an earlier question of yours, you cannot achieve that because in your case, you should pass 2 different queries (first and second) to a single adapter, which is not possible with FirestoreRecyclerAdapter. You can either use the first query or the second with a single instance of your adapter.

A solution would be to create two different lists, containing the results from each query and combine them. Then you can pass the resulting list to another adapter, let's say an ArrayAdapter and then display the results into a ListView or even better in a RecyclerView. The problem in this case is that you will not be able to use the real-time features that the FirestoreRecyclerAdapter class provides but this approach will solve your problem.

Edit:

According to your request from the comment section, I'll give you an example on how to paginate a query on button click in the easiest way, using a ListView and an ArrayAdapter. You can achieve the same thing also using a RecyclerView when scrolling down. But to keep things simple, let's assume we have a ListView and a Button and we want to load more items to the list on every button click. For that, let's define first the views :

ListView listView = findViewById(R.id.list_view);
Button button = findViewById(R.id.button);

Let's assume we have a database structure that looks like this:

Firestore-root
   |
   --- products (collection)
         |
         --- productId (document)
                |
                --- productName: "Product Name"

And a model class that looks like this:

public class ProductModel {
    private String productName;

    public ProductModel() {}

    public ProductModel(String productName) {this.productName = productName;}

    public String getProductName() {return productName;}

    @Override
    public String toString() { return productName; }
}

Now, let's define a query with the limit set to 3.

FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference productsRef = rootRef.collection("products");
Query firstQuery = productsRef.orderBy("productName", Query.Direction.ASCENDING).limit(3);

This means that on every button click, we'll load 3 more items. And now, here is the code that does the magic:

firstQuery.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
    @Override
    public void onComplete(@NonNull Task<QuerySnapshot> task) {
        if (task.isSuccessful()) {
            List<ProductModel> list = new ArrayList<>();
            for (DocumentSnapshot document : task.getResult()) {
                ProductModel productModel = document.toObject(ProductModel.class);
                list.add(productModel);
            }
            ArrayAdapter<ProductModel> arrayAdapter = new ArrayAdapter<>(context, android.R.layout.simple_list_item_1, list);
            listView.setAdapter(arrayAdapter);
            lastVisible = task.getResult().getDocuments().get(task.getResult().size() - 1);

            button.setOnClickListener(new View.OnClickListener() {
                @Override
                public void onClick(View view) {
                    Query nextQuery = productsRef.orderBy("productName", Query.Direction.ASCENDING).startAfter(lastVisible).limit(3);
                    nextQuery.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
                        @Override
                        public void onComplete(@NonNull Task<QuerySnapshot> t) {
                            if (t.isSuccessful()) {
                                for (DocumentSnapshot d : t.getResult()) {
                                    ProductModel productModel = d.toObject(ProductModel.class);
                                    list.add(productModel);
                                }
                                arrayAdapter.notifyDataSetChanged();
                                lastVisible = t.getResult().getDocuments().get(t.getResult().size() - 1);
                            }
                        }
                    });
                }
            });
        }
    }
});

In which lastVisible is a DocumentSnapshot object that represents the last visibile item from the query. In this case, every third one and it is declared as gloabl variable:

private DocumentSnapshot lastVisible;

Edit2:
Here you have also the solution on how you can get the data from your Firestore database and display it in smaller chunks in a RecyclerView when user scrolls.

Firestore query result pagination

How to run a query for a large set of documents that is likely to exceed the single query limit?

The best solution I can think of is to create an alternative structure and duplicate some data. Since a query of 1000 results might exceed the limit, you should consider creating another collection that looks like this:

Firebase-root
  |
  --- revenues (collection)
       |
       --- May 2021 (document)
       |    |
       |    --- [{May 17, 2021 at 7:51:42 PM UTC+3: 123.33}, 
       |         {May 18, 2021 at 3:36:11 PM UTC+3: 444.74}]
       |
       --- June 2021 (document)
            |
            --- [{June 11, 2021 at 3:12:22 PM UTC+3: 523.18}, 
                 {June 23, 2021 at 2:39:54 PM UTC+3: 253.14}]

As you can see, I have created a new collection called "revenues" that contains a document for each month of the year. Each document contains an array, that contains in terms key values pairs, where the key is the date and the value is the total amount of the invoice. As you can see, in my example above I have added in each month only two invoices. However, if you'll only store the above data (date/amount), you'll be able to store even much more than 1000 invoices within a single array, meaning that you'll be able to stay below the 1MiB max document size limit.

Since the user selects the start and the end date, you'll always know which document to start with and which to document to end with. So in this solution, you'll calculate the revenue on the client. For example, if you want to calculate the entire revenue from May 17th, 2021 to June 23th 2021, you have to read both documents and sum only the elements that exist between those dates. Instead of reading 1000 documents to get the total revenue, which in my opinion is a little costly, you'll only need to read two documents, assuming that a document might hold 1000 invoices.

This practice is also called denormalization and is a common practice when it comes to Firebase. If you are new to NoSQL databases, I recommend you see this video, Denormalization is normal with the Firebase Database for a better understanding. It is for Firebase Realtime Database but the same rules apply to Cloud Firestore.

Also, when you are duplicating data, there is one thing that you need to keep in mind. In the same way, you are adding data, you need to maintain it. In other words, if you want to update/delete an invoice, you need to do it in every place that it exists.

For more information please also see my answer from the following post:

• What is denormalization in Firebase Cloud Firestore?

If you want to directly map an array of objects from Cloud to a List of custom objects, please check my article from the following URL:

• How to map an array of objects from Cloud Firestore to a List of objects?

If you also want to implement pagination, please check my answer from the following post:

• How to paginate Firestore with Android?

In which you can paginate queries by combining query cursors with the limit() method. It's some kind of old but I also recommend you take a look at this video for a better understanding.

If you need pagination on button click, please my answer below:

• Is there a way to paginate queries by combining query cursors using FirestoreRecyclerAdapter?

How to paginate Firestore with Android?

As it is mentioned in the official documentation, the key for solving this problem is to use the startAfter() method. So you can paginate queries by combining query cursors with the limit() method. You'll be able to use the last document in a batch as the start of a cursor for the next batch.

To solve this pagination problem, please see my answer from this post, in which I have explained step by step, how you can load data from a Cloud Firestore database in smaller chunks and display it in a ListView on button click.

Solution:

To get the data from your Firestore database and display it in smaller chunks in a RecyclerView, please follow the steps below.

Let's take the above example in which I have used products. You can use products, cities or whatever you want. The principles are the same. Assuming that you want to load more products when user scrolls, I'll use RecyclerView.OnScrollListener.

Let's define first the RecyclerView, set the layout manager to LinearLayoutManager and create a list. We also instantiate the adapter using the empty list and set the adapter to our RecyclerView:

RecyclerView recyclerView = findViewById(R.id.recycler_view);
recyclerView.setLayoutManager(new LinearLayoutManager(this));
List<ProductModel> list = new ArrayList<>();
ProductAdapter productAdapter = new ProductAdapter(list);
recyclerView.setAdapter(productAdapter);

Let's assume we have a database structure that looks like this:

Firestore-root
   |
   --- products (collection)
         |
         --- productId (document)
                |
                --- productName: "Product Name"

And a model class that looks like this:

public class ProductModel {
    private String productName;

    public ProductModel() {}

    public ProductModel(String productName) {this.productName = productName;}

    public String getProductName() {return productName;}
}

This how the adapter class should look like:

private class ProductAdapter extends RecyclerView.Adapter<ProductViewHolder> {
    private List<ProductModel> list;

    ProductAdapter(List<ProductModel> list) {
        this.list = list;
    }

    @NonNull
    @Override
    public ProductViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
        View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.item_product, parent, false);
        return new ProductViewHolder(view);
    }

    @Override
    public void onBindViewHolder(@NonNull ProductViewHolder productViewHolder, int position) {
        String productName = list.get(position).getProductName();
        productViewHolder.setProductName(productName);
    }

    @Override
    public int getItemCount() {
        return list.size();
    }
}

The item_product layout contains only one view, a TextView.

<TextView
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:id="@+id/text_view"
    android:textSize="25sp"/>

And this is how the holder class should look like:

private class ProductViewHolder extends RecyclerView.ViewHolder {
    private View view;

    ProductViewHolder(View itemView) {
        super(itemView);
        view = itemView;
    }

    void setProductName(String productName) {
        TextView textView = view.findViewById(R.id.text_view);
        textView.setText(productName);
    }
}

Now, let's define a limit as a global variable and set it to 15.

private int limit = 15;

Let's define now the query using this limit:

FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference productsRef = rootRef.collection("products");
Query query = productsRef.orderBy("productName", Query.Direction.ASCENDING).limit(limit);

Here is the code that also does the magic in your case:

query.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
    @Override
    public void onComplete(@NonNull Task<QuerySnapshot> task) {
        if (task.isSuccessful()) {
            for (DocumentSnapshot document : task.getResult()) {
                ProductModel productModel = document.toObject(ProductModel.class);
                list.add(productModel);
            }
            productAdapter.notifyDataSetChanged();
            lastVisible = task.getResult().getDocuments().get(task.getResult().size() - 1);

            RecyclerView.OnScrollListener onScrollListener = new RecyclerView.OnScrollListener() {
                @Override
                public void onScrollStateChanged(RecyclerView recyclerView, int newState) {
                    super.onScrollStateChanged(recyclerView, newState);
                    if (newState == AbsListView.OnScrollListener.SCROLL_STATE_TOUCH_SCROLL) {
                        isScrolling = true;
                    }
                }

                @Override
                public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
                    super.onScrolled(recyclerView, dx, dy);

                    LinearLayoutManager linearLayoutManager = ((LinearLayoutManager) recyclerView.getLayoutManager());
                    int firstVisibleItemPosition = linearLayoutManager.findFirstVisibleItemPosition();
                    int visibleItemCount = linearLayoutManager.getChildCount();
                    int totalItemCount = linearLayoutManager.getItemCount();

                    if (isScrolling && (firstVisibleItemPosition + visibleItemCount == totalItemCount) && !isLastItemReached) {
                        isScrolling = false;
                        Query nextQuery = productsRef.orderBy("productName", Query.Direction.ASCENDING).startAfter(lastVisible).limit(limit);
                        nextQuery.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
                            @Override
                            public void onComplete(@NonNull Task<QuerySnapshot> t) {
                                if (t.isSuccessful()) {
                                    for (DocumentSnapshot d : t.getResult()) {
                                        ProductModel productModel = d.toObject(ProductModel.class);
                                        list.add(productModel);
                                    }
                                    productAdapter.notifyDataSetChanged();
                                    lastVisible = t.getResult().getDocuments().get(t.getResult().size() - 1);

                                    if (t.getResult().size() < limit) {
                                        isLastItemReached = true;
                                    }
                                }
                            }
                        });
                    }
                }
            };
            recyclerView.addOnScrollListener(onScrollListener);
        }
    }
});

In which lastVisible is a DocumentSnapshot object which represents the last visible item from the query. In this case, every 15'th one and it is declared as a global variable:

private DocumentSnapshot lastVisible;

And isScrolling and isLastItemReached are also global variables and are declared as:

private boolean isScrolling = false;
private boolean isLastItemReached = false;

---

If you want to get data in realtime, then instead of using a get() call you need to use addSnapshotListener() as explained in the official documentation regarding listening to multiple documents in a collection. More information you can find the following article:

• How to create a clean Firestore pagination with real-time updates?

how to add Firestore pagination in android

This is the easiest way in which you can paginate a query using a ListView and an ArrayAdapter on button click. A more complex approach would be found in this example where I have explained how you can paginate a query when user scrolls using a RecyclerView and a custom adapter.

In both examples, the key for solving the problem is to use the startAfter() method. So you can paginate queries by combining query cursors with the limit() method. You need to use the last document in a batch as the start of a cursor for the next batch.

If you'll prefer to use the second approach, I also recommend you take a look at this video for a better understanding.

If you want a solution for a real-time pagination, please check the following article:

• How to create a clean Firestore pagination with real-time updates?

---

Related Topics