More Efficient Way to Retrieve Firebase Data

More efficient way to retrieve Firebase Data?

One of the golden rules of Firebase is to only nest your data when you always want to retrieve all of it. The reason for this rule is that Firebase always returns a complete node. You cannot partially retrieve some of the data in that node and not other data.

The Firebase guide on structuring data, says this about it:

Because we can nest data up to 32 levels deep, it's tempting to think that this should be the default structure. However, when we fetch data at a location in our database, we also retrieve all of its child nodes. Therefore, in practice, it's best to keep things as flat as possible, just as one would structure SQL tables.

You should really read that entire section of the docs, since it contains some pretty good examples.

In your case, you'll need to modify your data structure to separate the event attendees from the event metadata:

events
  -JFSDFHdsf89498432
    eventCreator: "Stephen"
    eventCreatorId: 1764137
  -JOeDFJHFDSHJ14312
    eventCreator: "puf"
    eventCreatorId: 892312
event_attendees
  -JFSDFHdsf89498432
    -JSAJKAS75478
      name: "Johnny Appleseed"
    -JSAJKAS75412
      name: "use1871869"
  -JOeDFJHFDSHJ14312
    -JaAasdhj1382
      name: "Frank van Puffelen"
    -Jo1asd138921
      name: "use1871869"

This way, you can retrieve the event metadata without retrieving the attendees and vice versa.

What is the best way to retrieve data from Firebase realtime database to ListView in Android? ... Closed

You're missing a call to listViewAdapter.notifyDataSetChanged() in your onDataChange method. Without such a call, Android doesn't know that it needs to repaint the list view.

Every time you modify the data set that the list view shows, you need to notify the adapter of this. In your case that should be as simple as:

arrayList.add(listViewHelperClass);
listViewAdapter.notifyDataSetChanged();

Most efficient way to fetch data from Firebase

It sounds like you currently remove your observers when the view disappears. At that point Firebase will remove the users data from its memory cache. So indeed when you re-attach the listener in viewWillAppear, the data will need to be reloaded from the Firebase servers.

If you don't want the data to be reloaded, you'll need to make sure that it remains available on the device. There are two options for this:

Ensure the data stays in memory
Ensure the data is persisted to disk

Ensure the data stays in memory

To ensure the data stays in memory, you must keep the observer attached even after the user navigates away from the view. This means that you'll want another class to manage the observer, one that doesn't remove the observer when the user navigates.

The simplest approach for this is to have a global singleton that manages the observers and the user data, e.g. UserDataManager. When your view first calls this class, create/attach the listener and read the user data. When a subsequent view needs the users, they'll already be loaded in the UserDataManager ready for use.

The main problem with this is that as you do this for more data, you may end up keeping more data in memory than needed. So be aware of memory consumption of such unneeded data, and prune (remove listeners) as needed.

Ensure the data is persisted to disk

A simple way to reduce the amount of data that is downloaded is to =enable disk persistence](https://firebase.google.com/docs/database/ios/offline-capabilities#section-disk-persistence) when the app starts.

If disk persistence is enabled, the Firebase client stores data that it recently observed to disk (in addition to keeping any active data in memory). When you attach a new observer and the data isn't in memory yet, Firebase will check the disk cache for that data. If the data exists in the disk cache, it uses that to rebuild its memory cache and performs a much cheaper (bandwidth wise) check against the server.

Flutter most effective way to retrieve data from firebase in a Tab Navigation

There's Timestamp field in Firebase and you can use it for calculating whether the data is in a given boundary.

you can add timestamp value to your data:

Timestamp.now()

for example,

 Timestamp get timeLimit {
    final limit = DateTime.now().subtract(const Duration(days: 1));
    return Timestamp.fromDate(limit);
  }

you can get the lower limit of timestamp value that is 1day from the moment when you query.

Then

QuerySnapshot snapshot = await db
        .collection('data')
        .where('timestamp', isGreaterThanOrEqualTo: timeLimit)
        .limit(30)

Best way to retrieve Firebase data and return it, or an alternative way

To separate the code out, you'd pass in a callback:

function getTwoLatestLocations(callback)
{
    var twoLocations;
    myFirebaseRef.limitToLast(2).once('value', function (dataSnapshot) {
        twoLocations = dataSnapshot.val();
        callback(twoLocations);
    }, function (errorObject) {
        // code to handle read error
        console.log("The read failed: " + errorObject.code);
    });

}

And then calling it like this:

function someCalcutationsWithTheTwoLastLocations()
{
    getTwoLatestLocations(function(twoLocations) {
        // Do some calculations
    });
}

The important thing to always remember is that you cannot wait for something that is loaded asynchronously.

This has been covered quite a few times, including in this top answer from the list of related questions: Handling Asynchronous Calls (Firebase) in functions

How to efficiently query from Firebase Realtime Database using query or specific path

I don't know which language (which Client SDK) you use in your app, but here is a solution with the JavaScript SDK.

Let's imagine you have a Relatime Database structure as follows

  "parentnode" : {
    "currencies" : {
      "EUR" : {
        "value" : {
          "price" : 201
        }
      },
      "USD" : {
        "value" : {
          "price" : 343
        }
      },
      "value" : {
        "EUR_price" : 201,
        "USD_price" : 343,
        "currency-source" : "fx"
      }
    }
  }

Under a parentnode you have a currencies node which corresponds to the examples in your question.

In case you want to listen to /currencies/<currency>/value, you would do as follows:

  var db = firebase.database();

  var currency = 'EUR';

  var ref = db.ref().child('parentnode/currencies/' + currency);
  ref.on('value', function(snapshot) {
    var data = snapshot.val();
    console.log(data);
  });

In case you want to listen to /currencies/value/<currency>_price and get the price and the currency-source values, you would do as follows:

  var db = firebase.database();

  var currency = 'EUR';

  var ref = db.ref().child('parentnode/currencies/value');
  ref.on('value', function(snapshot) {
    var data = snapshot.val();
    var price = data[currency + '_price'];
    var source = data['currency-source'];
    console.log(price);
    console.log(source);
  });

As mentioned in your comment, the second approach implies that "we will download all the data leaf under /currencies/value/".

I can think of two other possible approaches. Choosing one over the other really depends on your functional requirements, i.e. what you do with these values in your front-end.

1/ Set two listeners

The idea is to set one listener for 'parentnode/currencies/value/' + currency + '_price' and one for 'parentnode/currencies/value/currency-source', as follows:

  var currency = 'EUR';

  var ref2 = db
    .ref()
    .child('parentnode/currencies/value/' + currency + '_price');
  ref2.on('value', function(snapshot) {
    var data = snapshot.val();
    console.log(data);
  });

  var ref3 = db.ref().child('parentnode/currencies/value/currency-source');
  ref3.on('value', function(snapshot) {
    var data = snapshot.val();
    console.log(data);
  });

2/ Query the `currency-source` value within the listener

With this second approach, in the listener to 'parentnode/currencies/value/' + currency + '_price', we query the database with the once() method to get the value of currency-source:

  var ref4 = db
    .ref()
    .child('parentnode/currencies/value/' + currency + '_price');
  ref4.on('value', function(snapshot) {
    var data = snapshot.val();
    console.log(data);
    db.ref()
      .child('parentnode/currencies/value/currency-source')
      .once('value')
      .then(function(dataSnapshot) {
        console.log(dataSnapshot.val());
      });
  });

Note that if you do not need to set a listener, i.e. you want to fetch the data once (e.g. by triggering the fetch from a button, or on page loading, etc..) you should only use the once() method and then you can chain the two calls as follows:

  var currency = 'EUR';
  var ref5 = db.ref().child('parentnode/currencies/value/' + currency + '_price');
  var ref6 = db.ref().child('parentnode/currencies/value/currency-source');

  ref5
    .once('value')
    .then(function(dataSnapshot) {
      console.log(dataSnapshot.val());
      return ref6.once('value');
    })
    .then(function(dataSnapshot) {
      console.log(dataSnapshot.val());
    });

More Efficient Way to Retrieve Firebase Data