Convert Django Model Object to Dict with All of the Fields Intact

Convert Django Model object to dict with all of the fields intact

There are many ways to convert an instance to a dictionary, with varying degrees of corner case handling and closeness to the desired result.

---

1. instance.__dict__

instance.__dict__

which returns

{'_foreign_key_cache': <OtherModel: OtherModel object>,
 '_state': <django.db.models.base.ModelState at 0x7ff0993f6908>,
 'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key_id': 2,
 'id': 1,
 'normal_value': 1,
 'readonly_value': 2}

This is by far the simplest, but is missing many_to_many, foreign_key is misnamed, and it has two unwanted extra things in it.

---

2. model_to_dict

from django.forms.models import model_to_dict
model_to_dict(instance)

which returns

{'foreign_key': 2,
 'id': 1,
 'many_to_many': [<OtherModel: OtherModel object>],
 'normal_value': 1}

This is the only one with many_to_many, but is missing the uneditable fields.

---

3. model_to_dict(..., fields=...)

from django.forms.models import model_to_dict
model_to_dict(instance, fields=[field.name for field in instance._meta.fields])

which returns

{'foreign_key': 2, 'id': 1, 'normal_value': 1}

This is strictly worse than the standard model_to_dict invocation.

---

4. query_set.values()

SomeModel.objects.filter(id=instance.id).values()[0]

which returns

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key_id': 2,
 'id': 1,
 'normal_value': 1,
 'readonly_value': 2}

This is the same output as instance.__dict__ but without the extra fields.
foreign_key_id is still wrong and many_to_many is still missing.

---

5. Custom Function

The code for django's model_to_dict had most of the answer. It explicitly removed non-editable fields, so removing that check and getting the ids of foreign keys for many to many fields results in the following code which behaves as desired:

from itertools import chain

def to_dict(instance):
    opts = instance._meta
    data = {}
    for f in chain(opts.concrete_fields, opts.private_fields):
        data[f.name] = f.value_from_object(instance)
    for f in opts.many_to_many:
        data[f.name] = [i.id for i in f.value_from_object(instance)]
    return data

While this is the most complicated option, calling to_dict(instance) gives us exactly the desired result:

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

---

6. Use Serializers

Django Rest Framework's ModelSerializer allows you to build a serializer automatically from a model.

from rest_framework import serializers
class SomeModelSerializer(serializers.ModelSerializer):
    class Meta:
        model = SomeModel
        fields = "__all__"

SomeModelSerializer(instance).data

returns

{'auto_now_add': '2018-12-20T21:34:29.494827Z',
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

This is almost as good as the custom function, but auto_now_add is a string instead of a datetime object.

---

Bonus Round: better model printing

If you want a django model that has a better python command-line display, have your models child-class the following:

from django.db import models
from itertools import chain

class PrintableModel(models.Model):
    def __repr__(self):
        return str(self.to_dict())

    def to_dict(instance):
        opts = instance._meta
        data = {}
        for f in chain(opts.concrete_fields, opts.private_fields):
            data[f.name] = f.value_from_object(instance)
        for f in opts.many_to_many:
            data[f.name] = [i.id for i in f.value_from_object(instance)]
        return data

    class Meta:
        abstract = True

So, for example, if we define our models as such:

class OtherModel(PrintableModel): pass

class SomeModel(PrintableModel):
    normal_value = models.IntegerField()
    readonly_value = models.IntegerField(editable=False)
    auto_now_add = models.DateTimeField(auto_now_add=True)
    foreign_key = models.ForeignKey(OtherModel, related_name="ref1")
    many_to_many = models.ManyToManyField(OtherModel, related_name="ref2")

Calling SomeModel.objects.first() now gives output like this:

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

Django: Converting an entire set of a Model's objects into a single dictionary

Does this need to create an actual dict? could you get by with only something that looked like a dict?

class DictModelAdaptor():
    def __init__(self, model):
        self.model = model

    def __getitem__(self, key):
        return self.model.objects.get(key=key)

    def __setitem__(self, key, item):
        pair = self.model()
        pair.key = key
        pair.value = item
        pair.save()

    def __contains__(self, key):
        ...

You could then wrap a model in this way:

modelDict = DictModelAdaptor(DictModel)
modelDict["name"] = "Bob Jones"

etc...

django convert model instance to dict

This actually already exists in Django, but its not widely documented.

from django.forms import model_to_dict
my_obj = User.objects.first()
model_to_dict(my_obj,
    fields = [...], # fields to include
    exclude =  [...], # fields to exclude
)

Django convert model objects to dictionary in large volumes results in server timeout

At first glance, your main problem is most likely you're hitting the database twice for each EntryState instance.

convertToDict method makes use of FK entry and for each entry, you also fetch the M2M tags. Solution is to optimize the query.

First, let's identify the problem.
When you put this code before the end of the view, you'll see how many times you're hitting the database in the console.

def myView(request):
    ....
    from django.db import connection; qq=connection.queries; print(f'QUERY: {len(qq)}')
    return render(request, 'my_template.html', context)

Now, try to improve the query in the view.

query = EntryState.objects.filter(field1 < 10)  # Assuming this is a valid query that uses something like `field1__lt=10`

You can use select_related for the Entry FK so that it'll be retrieved within that single query (Saves N hits to the DB)

query = EntryState.objects.filter(field1__lt=10).select_related('entry')

Also you can add prefetch related to retrieve the tags for ALL entries in one hit.

query = EntryState.objects.filter(field1__lt=10).select_related('entry').prefetch_related('entry__tags_set')

After each change, you can see how many times you hit to the database so that you can see the problem getting fixed. You can check the number of DB hits again to ensure it's optimized.

For more info, please read about query optimization, select_related and prefetch_related.

You can also use some apps to monitor the performance of your views: django-silk, django-debug-toolbar

Source: https://docs.djangoproject.com/en/4.0/ref/models/querysets/#select-related

expanding dict model field in django

Option 1, solution with JSONField:

class AJsonModel(models.Model):
    name = models.CharField(null=True, max_length=255)
    data = models.JSONField(null=True)

    def __str__(self):
        return f"A list of {[n for n in self.data]}" 

data = [10, 20, 30, 40, 50]
dict_data = {k:v for (k, v) in enumerate(data, 1)}

m = AJsonModel.objects.create(name="One", data=dict_data)
m.save()

Unfortunately for this method, you will have to do some work everytime you want to add another value to the list:

c = AJsonModel.objects.get(id=1)
values = list(c.data.values())
values.append(60)
c.data = {k:v for (k, v) in enumerate(values, 1)}
c.save()

Option 2, use related table:

class AJsonModelWithChild(models.Model):
    name = models.CharField(null=True, max_length=255)

    def __str__(self):
        return f"{self.name}"

class GrowingChild(models.Model):
    value = models.IntegerField(default=0)
    parent = models.ForeignKey(AJsonModelWithChild, on_delete=models.CASCADE)

    def __str__(self):
        return f"{self.id}: {self.value}"

The good thing for this method is that you do not have to think about the other, previous values stored before - just add a new child for this parent:

parent = AJsonModelWithChild.objects.create(name="Two")
parent.save()
child = GrowingChild.objects.create(value=100, parent=parent)
child.save()

How can I convert django model object and related object in to list of dictionary?

q_customer = Customer.objects.all()
L = []
for customer in q_custmer:
  Dcustomer = {'name':customer.name,
         'Payment':[{'amount',x.amount} for x in customer.payment_set]
  L.append[D]

# L now holds the list

How do you add one object to a Django Model that has a ManyToManyField?

The * in the arguments converts a list to individual args. For example-

lst = [1,2,3,4,5]

function(*lst)

can be just read as

function(1,2,3,4,5)

You have used follower.followers.add(*user). Hence, user must be an iterable to be unpacked and passed as a list or arguments. But user is a single User object.

You should just use follower.followers.add(user) in this case.

Django Project : 'dict' object has no attribute 'code'

Classic issue I'm also having when switching over eg from Javascript :-)
There is no dot notation in python to access dicts. There are basically 2 way to access a dict in python:

product["code"]

or

product.get("code")

The latter has the advantage that you can pass in a default value in case the key "code" is not available:

product.get("code", "123421")

---

Related Topics