How to paginate Firestore with Android?
As it is mentioned in the official documentation, the key for solving this problem is to use the startAfter() method. So you can paginate queries by combining query cursors with the limit() method. You'll be able to use the last document in a batch as the start of a cursor for the next batch.
To solve this pagination problem, please see my answer from this post, in which I have explained step by step, how you can load data from a Cloud Firestore database in smaller chunks and display it in a ListView on button click.
Solution:
To get the data from your Firestore database and display it in smaller chunks in a RecyclerView, please follow the steps below.
Let's take the above example in which I have used products. You can use products, cities or whatever you want. The principles are the same. Assuming that you want to load more products when user scrolls, I'll use RecyclerView.OnScrollListener.
Let's define first the RecyclerView, set the layout manager to LinearLayoutManager and create a list. We also instantiate the adapter using the empty list and set the adapter to our RecyclerView:
RecyclerView recyclerView = findViewById(R.id.recycler_view);
recyclerView.setLayoutManager(new LinearLayoutManager(this));
List<ProductModel> list = new ArrayList<>();
ProductAdapter productAdapter = new ProductAdapter(list);
recyclerView.setAdapter(productAdapter);
Let's assume we have a database structure that looks like this:
Firestore-root
|
--- products (collection)
|
--- productId (document)
|
--- productName: "Product Name"
And a model class that looks like this:
public class ProductModel {
private String productName;
public ProductModel() {}
public ProductModel(String productName) {this.productName = productName;}
public String getProductName() {return productName;}
}
This how the adapter class should look like:
private class ProductAdapter extends RecyclerView.Adapter<ProductViewHolder> {
private List<ProductModel> list;
ProductAdapter(List<ProductModel> list) {
this.list = list;
}
@NonNull
@Override
public ProductViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.item_product, parent, false);
return new ProductViewHolder(view);
}
@Override
public void onBindViewHolder(@NonNull ProductViewHolder productViewHolder, int position) {
String productName = list.get(position).getProductName();
productViewHolder.setProductName(productName);
}
@Override
public int getItemCount() {
return list.size();
}
}
The item_product layout contains only one view, a TextView.
<TextView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/text_view"
android:textSize="25sp"/>
And this is how the holder class should look like:
private class ProductViewHolder extends RecyclerView.ViewHolder {
private View view;
ProductViewHolder(View itemView) {
super(itemView);
view = itemView;
}
void setProductName(String productName) {
TextView textView = view.findViewById(R.id.text_view);
textView.setText(productName);
}
}
Now, let's define a limit as a global variable and set it to 15.
private int limit = 15;
Let's define now the query using this limit:
FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference productsRef = rootRef.collection("products");
Query query = productsRef.orderBy("productName", Query.Direction.ASCENDING).limit(limit);
Here is the code that also does the magic in your case:
query.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
@Override
public void onComplete(@NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
for (DocumentSnapshot document : task.getResult()) {
ProductModel productModel = document.toObject(ProductModel.class);
list.add(productModel);
}
productAdapter.notifyDataSetChanged();
lastVisible = task.getResult().getDocuments().get(task.getResult().size() - 1);
RecyclerView.OnScrollListener onScrollListener = new RecyclerView.OnScrollListener() {
@Override
public void onScrollStateChanged(RecyclerView recyclerView, int newState) {
super.onScrollStateChanged(recyclerView, newState);
if (newState == AbsListView.OnScrollListener.SCROLL_STATE_TOUCH_SCROLL) {
isScrolling = true;
}
}
@Override
public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
super.onScrolled(recyclerView, dx, dy);
LinearLayoutManager linearLayoutManager = ((LinearLayoutManager) recyclerView.getLayoutManager());
int firstVisibleItemPosition = linearLayoutManager.findFirstVisibleItemPosition();
int visibleItemCount = linearLayoutManager.getChildCount();
int totalItemCount = linearLayoutManager.getItemCount();
if (isScrolling && (firstVisibleItemPosition + visibleItemCount == totalItemCount) && !isLastItemReached) {
isScrolling = false;
Query nextQuery = productsRef.orderBy("productName", Query.Direction.ASCENDING).startAfter(lastVisible).limit(limit);
nextQuery.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
@Override
public void onComplete(@NonNull Task<QuerySnapshot> t) {
if (t.isSuccessful()) {
for (DocumentSnapshot d : t.getResult()) {
ProductModel productModel = d.toObject(ProductModel.class);
list.add(productModel);
}
productAdapter.notifyDataSetChanged();
lastVisible = t.getResult().getDocuments().get(t.getResult().size() - 1);
if (t.getResult().size() < limit) {
isLastItemReached = true;
}
}
}
});
}
}
};
recyclerView.addOnScrollListener(onScrollListener);
}
}
});
In which lastVisible is a DocumentSnapshot object which represents the last visible item from the query. In this case, every 15'th one and it is declared as a global variable:
private DocumentSnapshot lastVisible;
And isScrolling and isLastItemReached are also global variables and are declared as:
private boolean isScrolling = false;
private boolean isLastItemReached = false;
If you want to get data in realtime, then instead of using a get() call you need to use addSnapshotListener() as explained in the official documentation regarding listening to multiple documents in a collection. More information you can find the following article:
- How to create a clean Firestore pagination with real-time updates?
How can I paginate with other collection details in firestore android
To be able to perform a query based on a category and a region, you should first read their IDs. So assuming that a user selects "category name 200" and the "tokyo" for the region, in order to get the corresponding products, please use the following lines of code:
FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference categoryRef = rootRef.collection("Category");
CollectionReference regionRef = rootRef.collection("Region");
CollectionReference productRef = rootRef.collection("Product");
categoryRef.whereEqualTo("categoryName", "category name 200").get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
@Override
public void onComplete(@NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
for (QueryDocumentSnapshot document : task.getResult()) {
String categoryId = document.getString("categoryId");
regionRef.whereEqualTo("regionName", "tokyo").get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
@Override
public void onComplete(@NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
for (QueryDocumentSnapshot document : task.getResult()) {
String regionId = document.getString("regionId");
//Perform a query to get the products
productRef.whereEqualTo("categoryId", categoryId).whereEqualTo("regionId", regionId).get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
@Override
public void onComplete(@NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
for (QueryDocumentSnapshot document : task.getResult()) {
String productName = document.getString("productName");
Log.d(TAG, productName);
}
} else {
Log.d(TAG, "Error getting documents: ", task.getException());
}
}
});
}
} else {
Log.d(TAG, "Error getting documents: ", task.getException());
}
}
});
}
} else {
Log.d(TAG, "Error getting documents: ", task.getException());
}
}
});
Be sure to have unique categories and regions, so that each query returns a single result.
If you need to order the results, you can also chain an orderBy() call, to order the products according to a specific property. Be also aware that an index is required for such a query.
Please also see below a simple solution to paginate the results in Firestore:
- How to paginate Firestore with Android?
How to Use FirebaseFirestore with Paging 3 Library?
There are plenty of useful guides online. In general, you can set a stream of paged data from Firestore into a Recycler View and from there it is simply a matter of paginating each request.
This can be a lengthy and involved process, so I will link a recommended tutorial, its source, and the relevant youtube guide.
- Medium: https://medium.com/how-to-paginate-firestore-using-paging-3-on-android
- Github: https://github.com/alexmamo/FirestorePagination/
- Youtube: https://www.youtube.com/watch?v=4JOzozH5eOQ
how to add Firestore pagination in android
This is the easiest way in which you can paginate a query using a ListView and an ArrayAdapter on button click. A more complex approach would be found in this example where I have explained how you can paginate a query when user scrolls using a RecyclerView and a custom adapter.
In both examples, the key for solving the problem is to use the startAfter() method. So you can paginate queries by combining query cursors with the limit() method. You need to use the last document in a batch as the start of a cursor for the next batch.
If you'll prefer to use the second approach, I also recommend you take a look at this video for a better understanding.
If you want a solution for a real-time pagination, please check the following article:
- How to create a clean Firestore pagination with real-time updates?
Firestore query result pagination
How to run a query for a large set of documents that is likely to exceed the single query limit?
The best solution I can think of is to create an alternative structure and duplicate some data. Since a query of 1000 results might exceed the limit, you should consider creating another collection that looks like this:
Firebase-root
|
--- revenues (collection)
|
--- May 2021 (document)
| |
| --- [{May 17, 2021 at 7:51:42 PM UTC+3: 123.33},
| {May 18, 2021 at 3:36:11 PM UTC+3: 444.74}]
|
--- June 2021 (document)
|
--- [{June 11, 2021 at 3:12:22 PM UTC+3: 523.18},
{June 23, 2021 at 2:39:54 PM UTC+3: 253.14}]
As you can see, I have created a new collection called "revenues" that contains a document for each month of the year. Each document contains an array, that contains in terms key values pairs, where the key is the date and the value is the total amount of the invoice. As you can see, in my example above I have added in each month only two invoices. However, if you'll only store the above data (date/amount), you'll be able to store even much more than 1000 invoices within a single array, meaning that you'll be able to stay below the 1MiB max document size limit.
Since the user selects the start and the end date, you'll always know which document to start with and which to document to end with. So in this solution, you'll calculate the revenue on the client. For example, if you want to calculate the entire revenue from May 17th, 2021 to June 23th 2021, you have to read both documents and sum only the elements that exist between those dates. Instead of reading 1000 documents to get the total revenue, which in my opinion is a little costly, you'll only need to read two documents, assuming that a document might hold 1000 invoices.
This practice is also called denormalization and is a common practice when it comes to Firebase. If you are new to NoSQL databases, I recommend you see this video, Denormalization is normal with the Firebase Database for a better understanding. It is for Firebase Realtime Database but the same rules apply to Cloud Firestore.
Also, when you are duplicating data, there is one thing that you need to keep in mind. In the same way, you are adding data, you need to maintain it. In other words, if you want to update/delete an invoice, you need to do it in every place that it exists.
For more information please also see my answer from the following post:
- What is denormalization in Firebase Cloud Firestore?
If you want to directly map an array of objects from Cloud to a List of custom objects, please check my article from the following URL:
- How to map an array of objects from Cloud Firestore to a List of objects?
If you also want to implement pagination, please check my answer from the following post:
- How to paginate Firestore with Android?
In which you can paginate queries by combining query cursors with the limit() method. It's some kind of old but I also recommend you take a look at this video for a better understanding.
If you need pagination on button click, please my answer below:
- Is there a way to paginate queries by combining query cursors using FirestoreRecyclerAdapter?
Android - Firestore infinite pagination with listeners
I have found mistake.
The working code is:
private void addMessagesEventListener(boolean firstTime) {
CollectionReference messagesCollection = chatsCollection.document(chat.getId()).collection(Constants.FIREBASE_MESSAGES_PATH);
Query query = messagesCollection.orderBy("timestamp", Query.Direction.DESCENDING);
if (!firstTime) {
query = query.startAt(startListen);
}
query.limit(20).get().addOnSuccessListener(queryDocumentSnapshots -> {
if (!firstTime) {
endListen = startListen;
}
startListen = queryDocumentSnapshots.getDocuments().get(queryDocumentSnapshots.size() - 1);
Query innerQuery = messagesCollection.orderBy("timestamp").startAt(startListen);
if(!firstTime) {
innerQuery = innerQuery.endBefore(endListen);
}
ListenerRegistration listener = innerQuery
.addSnapshotListener((queryDocumentSnapshots1, e) -> {
if (e != null) {
Log.w("SASA", "listen:error", e);
return;
}
for (DocumentChange dc : queryDocumentSnapshots1.getDocumentChanges()) {
Message message = dc.getDocument().toObject(Message.class);
switch (dc.getType()) {
case ADDED:
// add new message to list
messageListAdapter.addMessage(message);
if (firstTime) {
messagesList.smoothScrollToPosition(0);
}
break;
case REMOVED:
// remove message from list
messageListAdapter.removeMessage(message);
break;
}
}
});
listeners.add(listener);
});
}
The mistake was in query = query.startAt(startListen) and innerQuery = innerQuery.endBefore(endListen)
- startListen & endListen are of DocumentSnapshot type
- I am using sortedList in my adapter
You shoud add
private void detachListeners() {
for(ListenerRegistration registration : listeners) {
registration.remove();
}
}
in onDestroy to detach all listeners.
Code is listening for adding new messages and deleting old ones.
Related Topics
How to Efficiently Parse HTML With Java
Android, Getting Resource Id from String
How to Declare a Variable in Gradle Usable in Java
Java Using Much More Memory Than Heap Size (Or Size Correctly Docker Memory Limit)
The Use of Multiple Jframes: Good or Bad Practice
How to Append Text to an Existing File in Java
Stringbuilder VS String Concatenation in Tostring() in Java
What Causes and What Are the Differences Between Noclassdeffounderror and Classnotfoundexception
Immutability of Strings in Java
Javafx 8 Compatibility Issues - Fxml Static Fields
How to Paginate Firestore With Android
How to Set Httpresponse Timeout For Android in Java
Error: Could Not Find or Load Main Class
How to Write a Correct Micro-Benchmark in Java
How to Use Java.Util.Scanner to Correctly Read User Input from System.In and Act on It