Jeremy Stretch - Leveraging the ORM to enforce object level permissions

Summary

Jeremy Stretch's DjangoCon 2021 presentation introduces an advanced object-based permissions system for Django, developed for NetBox, a network infrastructure modeling application. The talk outlines the limitations of Django's default permissions and explores third-party packages before diving into a custom solution using JSON-stored QuerySet filters. This approach allows dynamic permissions based on object attributes, offering scalability and flexibility for diverse user needs. Stretch also discusses the implementation of a custom authentication backend and atomic transactions for secure object modifications, with plans to release the solution as a standalone package for the Django community.

Takeaways

• 😀 Jeremy Stretch, a former network engineer turned software developer, is the founder and lead maintainer of NetBox, an application for modeling network infrastructure.
• 🔧 NetBox is built on Django and has around 70 complex models with various relationships, making it an advanced application popular for managing large-scale network infrastructures.
• 🌐 NetBox's user base is diverse, with users having large networks and unique use cases, leading to the need for more robust permission systems beyond Django's native capabilities.
• 🛠️ Jeremy discussed the limitations of Django's default permissions system, which are based on models and do not support object-level permissions out of the box.
• 🔒 The talk focused on implementing object-based permissions in Django, allowing permissions to be granted for subsets of objects on a dynamic basis, even for objects not yet created.
• 📝 The default Django permissions are automatically created with four CRUD actions (view, add, change, delete) for each model, named in the format 'app_label.action_model_name'.
• 📦 Third-party packages like django-guardian and django-rules were considered for object-based permissions but did not meet NetBox's requirements due to scalability and flexibility issues.
• 💡 A custom solution was developed for NetBox that uses JSON to store QuerySet filters in the database, allowing for complex, attribute-based permissions.
• 🔄 The custom permissions system involves creating an ObjectPermission model that captures the flexibility of Django's ORM for permissions enforcement, using JSON to define constraints.
• 🔑 A custom authentication backend was implemented to support the new permissions logic, overriding Django's ModelBackend to handle object-level permissions evaluation.
• 🔍 The permissions are applied using a RestrictedQuerySet manager that simplifies the process of filtering objects based on user permissions and object attributes.

Q & A

What is the main topic of Jeremy Stretch's presentation at DjangoCon 2021?

Jeremy Stretch's presentation is about implementing object-based permissions in Django, focusing on how to grant permissions to users or groups for subsets of objects dynamically.

What is Jeremy Stretch's professional background before becoming a software developer?

Jeremy Stretch was a former network engineer before transitioning into full-time software development.

What is NetBox and why was it created?

NetBox is an application for modeling network infrastructure, including data center infrastructure and IP address management. It was conceived at DigitalOcean and released as an open source project under the Apache 2 license in 2016.

How does the diversity of NetBox's user base present challenges in terms of permissions?

The diversity of NetBox's user base, which includes users with large networks and unpredictable use cases across different industries and organizational sizes, presents challenges in creating a permissions system that can accommodate unique approaches and requirements.

What is the default permissions backend in Django and how does it work?

The default permissions backend in Django is ModelBackend, which automatically creates four default permissions (view, add, change, delete) for each model, following a specific naming structure.

Why did Jeremy Stretch and his team decide to build their own permissions system for NetBox?

They decided to build their own permissions system because existing packages like django-guardian and django-rules did not meet their needs for scalability and flexibility, especially considering the unpredictable and diverse use cases of NetBox users.

What is the key idea behind the custom permissions system developed for NetBox?

The key idea is to declare QuerySet filters in JSON, store them in the database, and use these filters to enforce permissions on objects based on their attributes, leveraging the flexibility of Django's ORM.

How does the ObjectPermission model differ from Django's stock Permission model?

The ObjectPermission model bundles model-based permissions together, allowing for more streamlined permission assignments, while the stock Permission model has one instance per content type per action.

What is the purpose of the RestrictedQuerySet manager in the custom permissions system?

The RestrictedQuerySet manager simplifies the process of applying permissions to a QuerySet by providing a .restrict method that takes the user and the desired action as parameters, handling the permissions retrieval and caching behind the scenes.

How does the custom permissions system handle the modification of objects to ensure they remain in a permitted state?

The system uses Atomic Transactions to check the pre-modified and post-modified states of an object. If the modified object no longer meets the permissions criteria, a custom exception is raised, undoing the transaction and preventing the changes from being saved to the database.

What is the current status of the custom permissions system developed for NetBox?

The custom permissions system has been implemented in NetBox version 2.10 and is being worked on to be extracted as a standalone package, expected to be available by the end of 2021 or early 2022.

Outlines

00:00

😀 Introduction to Object-Based Permissions in Django

Jeremy Stretch introduces his talk at DjangoCon 2021, focusing on implementing object-based permissions in Django. He shares his background as a former network engineer turned software developer and mentions his work at NS1 and as the founder of NetBox, an application for modeling network infrastructure. Jeremy highlights the diverse user base of NetBox and the challenges in managing permissions for various use cases and organizational structures. The talk aims to explore how to grant permissions dynamically to users or groups for subsets of objects within NetBox.

05:01

📚 Understanding Django's Default Permissions System

The speaker provides an overview of Django's default permissions framework, explaining the ModelBackend and the automatic creation of CRUD permissions for each model. He discusses the syntax and structure of permission names and how to evaluate permissions in views and templates. Jeremy also touches on the limitations of Django's built-in permissions when it comes to object-specific permissions, noting that the has_perm method does not support object-level permissions out of the box.

10:03

🔍 Exploring Third-Party Packages for Advanced Permissions

Jeremy reviews third-party packages like django-guardian and django-rules, which offer object-based permissions and flexible rule-based permissions, respectively. He explains how django-guardian assigns permissions to specific objects for users or groups, while django-rules allows developers to define predicates and rules for permissions enforcement. However, he concludes that these solutions do not fully meet the needs of NetBox due to their limitations in scalability and the need for static, predetermined rules.

15:05

🛠 Building a Custom Object-Based Permission System

After finding existing solutions inadequate, Jeremy describes the process of building a custom object-based permission system within NetBox. He outlines the design goals, which include compatibility with Django's stock permissions, flexibility in granting permissions based on object attributes, providing a user-friendly interface and API, and ensuring immediate application of permission changes. The custom solution leverages Django's ORM to filter objects by attributes and stores these filters in a new ObjectPermission model as JSON data.

20:07

🔄 Implementing QuerySet Filters for Permissions

The speaker details the implementation of the custom permission system, explaining how to apply QuerySet filters to enforce permissions. He describes the process of retrieving Object Permissions for a user, building a QuerySet with OR logic to combine constraints, and using this to filter objects. Jeremy also introduces the RestrictedQuerySet manager to simplify the application of permissions and discusses modifying the has_perm method to support the new system.

25:08

🚦 Handling Object Modifications with Atomic Transactions

Jeremy addresses the challenge of modifying objects while respecting permissions, explaining the use of atomic transactions to ensure that objects are not altered into non-permitted states. He outlines the steps for checking the pre- and post-modification states of an object and using exceptions to roll back changes that violate permissions. The speaker also highlights the need to secure all avenues for object modification, including APIs and bulk operations.

📘 Future Developments and Community Involvement

In the concluding part of the talk, Jeremy announces the ongoing work to extract the custom permission system from NetBox and prepare it as a standalone package for the Django community. He shares the timeline for the release and invites community members to contribute to the development and testing of the package. Jeremy also offers his availability for further questions and engagement on various platforms.

Keywords

💡Django

Django is a high-level Python web framework that encourages rapid development and clean, pragmatic design. In the context of the video, Django is the framework in which the speaker, Jeremy Stretch, is discussing the implementation of object-based permissions. The video revolves around enhancing Django's capabilities to handle complex permission scenarios, particularly in the NetBox application, which is built on Django.

💡Object-based permissions

Object-based permissions refer to a system where access rights are granted to users or groups for specific objects within an application, rather than just model-level permissions. In the video, Jeremy discusses the need for such permissions in NetBox to allow dynamic and granular control over what users can do with particular network infrastructure objects.

💡NetBox

NetBox is an open-source application for modeling network infrastructure, such as data center infrastructure and IP address management. It was mentioned as the application for which the speaker is implementing advanced permissions. The video emphasizes the complexity and diversity of NetBox's user base, which necessitates the development of robust permission systems.

💡ModelBackend

ModelBackend is the default permissions backend in Django, which automatically creates permissions for each model in the system. The video script discusses the limitations of ModelBackend when it comes to object-level permissions, leading to the exploration of alternative solutions.

💡CRUD

CRUD stands for Create, Read, Update, and Delete, which are the four basic operations that can be performed on data in a system. In the script, CRUD actions are related to the default permissions Django provides for each model: view, add, change, and delete, which correspond to the CRUD operations.

💡django-guardian

Django-guardian is a third-party package that provides object-level permissions in Django by allowing direct assignment of objects to specific users or groups. In the video, Jeremy mentions django-guardian as one of the solutions considered but ultimately not suitable for NetBox due to its requirement of explicit relationships.

💡django-rules

Django-rules is another third-party package that allows developers to define predicates and rules for permissions enforcement. It offers flexibility but relies on static and predetermined rules. Jeremy discusses it as an alternative to django-guardian, but it was also not a perfect fit for NetBox's needs due to the unpredictable use cases of its users.

💡QuerySet Filters

QuerySet Filters in Django are used to retrieve specific subsets of data from the database based on certain conditions. In the script, Jeremy describes how they implemented a custom permissions system using QuerySet Filters stored as JSON in the database, allowing for dynamic and flexible permission constraints.

💡Atomic Transactions

Atomic Transactions ensure that database operations are processed reliably without partial updates, which can be critical for maintaining data integrity. In the context of the video, Atomic Transactions are used to check the state of an object before and after modifications to ensure that changes do not violate the defined permissions.

💡Custom Authentication Backend

A Custom Authentication Backend in Django allows developers to override the default authentication process, including permission checks. Jeremy explains that they created a custom backend to support their new object-based permissions system, extending the built-in ModelBackend and overriding its methods to incorporate the new permission logic.

💡PermissionsViolation

PermissionsViolation is a custom exception used in the video's context to handle cases where a user attempts to perform an action that violates the object-based permissions rules. It is raised when an object's post-modification state no longer complies with the assigned permissions, and the transaction is rolled back to maintain the object's integrity.

Highlights

Jeremy Stretch, a former network engineer turned software developer, presents at DjangoCon 2021 on implementing object-based permissions in Django.

NetBox, an application for modeling network infrastructure, was conceived at DigitalOcean and released as an open-source project under the Apache 2 license in 2016.

NetBox, built on Django, has around 70 complex models with numerous relationships, making it an advanced application popular in recent years.

The diverse user base of NetBox includes large networks with tens or hundreds of thousands of nodes, presenting unique challenges in permissions management.

The standard Django permissions system, based on the ModelBackend, provides four default permissions (view, add, change, delete) for each model.

Django's has_perm method can evaluate permissions for a specific object, but the default ModelBackend returns false for any object other than none.

Packages like django-guardian and django-rules were considered for object-based permissions but did not meet the scalability and flexibility needs of NetBox.

Jeremy Stretch outlines the design goals for a custom permissions system in NetBox, emphasizing compatibility, flexibility, user-friendly interfaces, and immediate application of changes.

A new ObjectPermission model was created to store QuerySet filters as JSON in the database, allowing for dynamic permissions based on object attributes.

The ObjectPermission model correlates content types, models, users, groups, and actions, with optional constraints for more granular permissions.

A RestrictedQuerySet manager was developed to simplify the application of permissions to QuerySets, retrieving and caching permissions as needed.

A custom authentication backend was implemented to support the built-in permissions evaluation logic, overriding the default has_perm method.

Atomic transactions are used to ensure that objects cannot be modified into a non-permitted state, maintaining data integrity and permissions enforcement.

The permissions system is designed to work with bulk operations and API views, such as Django Rest Framework or GraphQL, ensuring comprehensive security.

Jeremy Stretch discusses the ongoing development of extracting the permissions system from NetBox into a standalone package for the Django community.

The standalone permissions package is expected to be available by the end of 2021 or early 2022, with contributions and community feedback encouraged.

Jeremy Stretch concludes the presentation by offering his availability for questions and future engagement on Twitter, Slack, or other channels.

Transcripts

- Hi everyone. This is Jeremy Stretch presenting

for DjangoCon 2021.

Today I'll be talking about implementing object based

permissions in Django. So let's go ahead and jump in.

First off a little bit about myself.

I am a former network engineer turned software developer.

So I originally got my start in my career in traditional

network infrastructure and engineering.

And over the past few years I've transitioned

into full-time software development.

Currently at the NS1, have been working with Django,

however, since around early 2008, I think,

when I originally had for, you know, just a personal blog.

These days though, I am probably best known

as the founder and lead maintainer of NetBox.

So what is NetBox?

It's an application for basically

modeling network infrastructure.

So if you think about traditional data center

infrastructure and IP address management,

performs all of those tasks, as well as some other roles.

It was originally conceived at DigitalOcean,

back when I was a network engineer

on the infrastructure team there,

and we released it as an open source project

under the Apache 2 license back in 2016.

So it's completely built on Django.

We've got about 70 or so complex models,

lots of relationships among them.

It's a pretty advanced application and it's gotten

really popular, especially in recent years.

If it sounds like something you're interested by all means,

check us out on GitHub.

But in developing NetBox,

we've noticed a few things about our users.

We have a very, very wide diverse user base.

Many of them have very, very large networks

to the tunes of tens or even hundreds of thousands

of nodes on a network or addresses on the network.

They also have unpredictable use cases, meaning, you know,

a financial company might use something different

than a service provider,

which would be different than, like,

higher education and so on.

So, we see wildly different use cases

among industries and different organizational sizes,

and depending on even what they're using it for.

Different organizational structures as well,

some people, some organizations out there

are a one or two person shops.

Some are huge teams running their infrastructure,

and every step along the way between those.

And so we've seen plenty of unique approaches around working

or working around limitations. So for example,

if an organization wants to model something

that NetBox doesn't support natively,

they have some clever work around sometimes

to make it do what they want it to.

This leads to some interesting challenges,

specifically around permissions.

So a couple of years back,

we started looking at, uh,

doing, seeing what we could do to provide

more robust permissions

in NetBox, beyond what Django provides natively

as a framework.

And really what the question boiled down to is just this:

It's how do we grant permissions to users

and or groups for subsets of objects

within NetBox on a dynamic basis?

Meaning we want them to apply,

we want the permissions to apply even to objects that

haven't been created yet.

So that's what I'm gonna be talking about today.

First, we'll start with a quick dive

into de Django permissions as they

exist in the framework today.

To provide some scope and context of this conversation.

If you're familiar with the acronym, triple A,

that's authentication authorization and accounting,

pretty standard term in security,

basically, we're going to be looking

at that second point right there.

So authentication is who you are.

Authorization is what you can do.

And accounting is what you did.

Permissions is pertinent to the second point is

authorization.

So we're not going to be talking about authentication

at all. Just, just the permissions framework.

The Default Permissions Backend

in Django is the ModelBackend

that's provided under cotrib.auth.backends.

Permissions are created automatically.

Every time we create a model and we get

four default permissions out of the box for each model.

Those are the,

what we'll refer to sometimes as CRUD actions being,

create, read, update, delete.

In Django vernacular those are going to be

view, add, change, and delete.

So the four basic options or operations or others

that you can do with an instance.

Permission names or follow the structure below here.

This is app label, dot, action, underscore,

name of the model.

Here's an example. We have a model called Book.

It's an application called Library.

It's got some fields on there and under the meta class,

we have permissions defined. So here we're actually,

we are defining a custom permission as well.

So Django is very flexible in that,

it lets you do this as well, pretty easily.

And the permission here is publish underscore book.

So this model will result in five

ultimate permissions being created.

Whenever you run the migration to introduce the model

within the Django application.

So you've got view book, add book,

change, book, delete book, and our custom permission,

which is published book.

When we evaluate these permissions,

if we're doing it in the view,

we're typically going to call it user dot has perm

or has perms with an S on the end

for evaluating multiple permissions within a template.

And a couple of ways we can do that.

Typically we're going to do it, perms dot

and then the permission name,

or look for the permission name

as a string within the perms context variable.

So none of this should be new.

Hopefully this is a quick refresher for folks who may

not have much use to mess

with permissions on a regular basis.

This is really, the kind of the crux of the matter,

everything we just talked about is for evaluating

permissions on a model basis. So for example,

you saw the book model, right?

And this applies to all instances,

all books that we might create in our Django application.

What if we want to enforce permissions

for a specific object?

Well, let's take a look at the has perm method.

Remember we call it user dos has perm

to evaluate that.

Within the doc string of this,

we see a note at the end.

So first off it has perm is going to just return true

if the user has the specified permission.

So I call has perm,

you know, view, underscore

or, sorry,

library dot view underscore book,

if they have that permission, it's gonna return it.

You'll note though that

the method signature has the option to pass an object

as well, but default it's going to be none.

And this line at the end

of the dock string here is kinda interesting.

So if an object is provided check permissions

for that object.

Now the documentation actually doesn't make any

mention of this.

So this is pretty interesting and encouraging.

So maybe, maybe this is already kind of built in,

let's see what we can do.

So starting from the has per method here

on the user instance,

we did a little bit of digging and it turns out

the rabbit hole goes pretty far.

The has perm method actually calls a utility function,

which references a method on the ModelBackend,

which in turn references,

a parent class of base Backend,

and it goes down a few more levels.

But ultimately what you'll find is that

it will return false, has perm, the initial method,

will return false because a ModelBackend dot get permissions

is saying, if you pass any object,

regardless of what's past there, if it's not none,

it's going to return an empty set,

because I don't know how, me being a ModelBackend is saying,

Hey, I don't know how to evaluate permissions

for a specific object. So I'm just going to say,

no, you don't have any. Right?

It's just feeling secure in that,

in that manner, which is great.

It's a good design choice. Unfortunately, it's,

not going to obviously give us anything to work with

out of the box.

All right. So what solutions are available if Django,

if it's not native in the framework?

The first package we looked at

you may be familiar with, is called django-guardian.

And it provides basically object-based permissions

through a direct assignment of objects

to specific users or groups.

So you can see from the code here,

what we're doing is basically

importing this user Object Permission.

And then we're going to assign this permission to a user

and a specific object.

And then we can evaluate that for a specific object.

And that's great. They've overridden that has perm method.

This is great if you have what I'll call

owner centric model.

So think about, like,

content management systems, blogs,

where the author of a piece of content should typically

be the owner. That's a great use case.

Unfortunately, it doesn't quite work for us

because we want to avoid those

explicit relationships just because of the scale and breadth

of which of resources that NetBox models.

It doesn't really scale for us.

Another package we looked at was django-rules.

So this is really cool in that it allows the developer

to define predicates and, and their terminology,

which can then be referenced to enforce permissions

or referenced by rules rather to enforce permissions.

So here's an example of a predicate being set,

is book author, which is basically some piece

of logic that will determine

whether or not a given user is an author of a book.

Now, obviously that's a very kind of trite example,

but you can imagine the flexibility as it forwards.

And then you have rules here that can be set and evaluated.

Tons of flexibility. Very cool.

Unfortunately it does rely on static

and predetermined rules, which weren't quite suitable

for our case, because as I alluded to earlier,

we don't really know how users

are going to use NetBox sometimes,

or how they're going to set up permissions

and what kind of assignments they're actually

going to be interested in evaluating.

So unfortunately that didn't quite work out for us either.

Can we build our own?

Nothing out there, at least I don't think

at the time suited to our needs.

So we set about trying to do it on our own.

Before we did that, we outlined some design goals.

So first off we wanted to make sure we retained

compatibility with a stock model-based permissions,

that you get out of Django and out of the box.

Second, we wanted to ensure that granting permissions

could be done based on arbitrary object attributes,

meaning we can say,

okay, you have, there's the concept of a site

in NetBox and say, you can assign users

to just have access to sites in the Europe region

or sites that have a status of active, for example.

We want to make it that

kind of to afford that degree of flexibility.

We also want to provide a user-friendly interface and an API

to make those assignments meaning that, you know,

we don't want users to have to go and edit code somewhere

to set up their rules for permission assignments.

We want that all to be done through the user interface so

that no one has to be a developer

just to assign permissions.

And finally, we want to make sure

permissions apply immediately upon changes.

And that's true in Django today.

Yeah, so we thought for some time about how best to do this.

And I think the aha moment really came about

when we realized this mechanism for filtering objects

by attributes, really already exists

in the form of the Django ORM.

That does exactly what we want.

Think about if we want to find,

if I want to find sites that are in Europe, I can do that.

I can filter by the region to which the site is assigned,

or I can filter by its status or,

or any number of other attributes.

I can traverse a related objects too.

So all that functionality is kind of there.

And we said, basically,

we just want a way to capture that flexibility of their ORM

during permissions enforcement.

So the idea was, let's declare QuerySet Filters in JSon,

and then just store them in the database.

So we're basically just storing QuerySet Filters.

Here's an example of that. So we've got JSon data there,

this is just a string at top,

we're filtering both region name, is Europe

and its status is active.

So this is a QuerySet Filter that applies the site model.

So what we're going to do is take that string,

which is just raw JSon,

load it using the loads string functionality

of the JSon library, into a dictionary called params.

This is now a Python dictionary.

Then we're going to pass that dictionary as a keyword

filters to the um, I'm sorry,

keyword arguments to the filter method

for the site objects QuerySet.

So pretty straightforward.

To do this though, we need a way to save

that JSon data in the database.

So the answer there of course, was to create a new model.

This is our ObjectPermission model,

or rather a very simplified version of it.

You can see some,

some fields that you probably would expect first off,

it has a name as a Boolean

to indicate whether or not it's enabled.

And then we have a bunch of many, too many fields.

So basically, we're mapping object types, groups

and users as many to many relationships.

And then we have an array field

on which we're

recording a set of actions that apply to this permission.

And we'll talk about that in a second.

And then finally at the bottom,

we see a JSon field called constraints.

And that is where we are in the previous example here

that the params, that's where we're saving

that in the database.

So basically this model is correlating content types

or models with users and or groups and actions.

So actions, again, going back to the view, add, change,

delete that are provided by default from Django,

and also, you can specify your custom actions if needed,

and they're stored as just an array of strings.

The constraints are optional.

If you don't provide constraints that essentially replicates

the built-in functionality of the Django permissions as they

exist today, just in a different manner.

But with constraints there,

basically we're doing a logical AND, when combining them

within an instance.

So for our previous example we had a region and status.

So it's going to be region is Europe AND status is active

because they're defined as one,

they're all defined as one set of constraints

in a specific Object Permission instance.

Whereas if we have multiple instances each

with their own constraints,

we're going to be doing a logical OR.

So if they were two different permissions,

we would say, you can, you know,

you have access to sites in Europe,

OR let's say any site that is active.

It's important to note that this model really supplants

Django's Stock Permission model.

So we're not, we're no longer using that.

And here they are side by side to give you a better idea

just of the differences.

So it's important to call out the Stock Permission model has

one instance per content type per action.

Whereas we kind of bundle all those up together

in ObjectPermission to make things

a little bit more streamlined.

Typically within NetBox, you know, we see most,

most of the right actions. So add change,

delete are all going to be bumbled together.

Not always the case, but where it is,

and that's going to be 90 plus percent of the time.

It's a little more efficient to do it in that manner.

It's also important to call out the, the groups and users,

many fields are actually on the Permission Object here,

whereas in a Stock model,

they're on the user and the group objects.

Okay. So we have our model here.

We have a way of conveying a QuerySet parameters,

or filters rather.

How do we actually apply them to a QuerySet?

So there's a few steps.

First, once we have these defined,

we're going to retrieve all of the Object Permissions

for a given user.

And we do that by filtering on both the user

or a group of which the user is a member

for the specific action that we care about.

And that double underscore contains,

is referencing the view action.

So here we care about view and of course we need

to make sure they are enabled.

That'll give us all instances of Object Permissions that

apply for this user ,

for the action that we care about.

Second, we're going to build a QuerySet,

that's filtered by all these constraints

using again, that OR logic.

So we start with a queue object and that allows us to do

complex filtering on the OR,

and basically iterate through these permissions

that we retrieved from the database

and build this constraints dictionary.

So for every constraint that we find,

every Object Permission instance, rather,

we are OR-ing its constraints.

So that all becomes this one big queue object

that we can then apply directly

to the filter method on a QuerySet.

So model the objects that filter,

and then just passing into strengths,

which again is a queue object. It's not a dictionary.

So we are just passing it directly.

Once we have that, if we need to check

for a specific object,

so that's going to give us all objects.

If we wanted to check for a specific object,

we just filter by the primary key.

So we're filtering by both primary key

and the assigned permissions.

So whether or not that primary key exists,

it's only going to return,

if the permissions actually apply to it.

So basically we can check,

we can call it dot first or dot exists to check

whether those permissions are in fact valid.

Obviously that's all a little bit unwieldy.

So we created a QuerySet manager called

RestrictedQuerySet. All this is doing is,

is providing a dot restrict method to just allow passing

the user and this desired action in when doing a query,

obviously simplifies that quite a bit.

And the permissions retrieval is actually being done

behind the scenes or it's being cashed.

Okay, so that kinda makes sense,

or hopefully it's starting to make sense.

What about user dot has perm? Right?

That's kind of where we started with all this.

What does that look like, with this implementation?

So it became apparent pretty quickly that we would need

a Custom Authentication Backend to support

the built-in permissions evaluation logic.

So what we did was extend Django's ModelBackend

meaning the name of the class and then override.

So we did two things.

First, we overrode the has perm method and we overrid

get all permissions.

And this was the one, if you remember from earlier,

this is the one that was actually responsible for returning

an empty set, whenever an object was passed.

So obviously we needed to remove, remove that check,

but yeah, the meat of it is under the has perm method.

So this method is really going to check a few things.

First off it the user's inactive,

we're going to return false.

If it's a super user we're going to return true,

then we're going to look for any model level permission

assigned to the user.

Because if you don't have a model of a level of permission,

we don't need the next step,

which is to check for the assignment

of specific constraints within those permissions.

So basically we're checking model level first,

if you don't have, for example,

if there are no permissions that will allow a user to mod-

to modify any site,

then we can go ahead and return false.

We don't have to check for the constraints.

But if we do find one or more,

then we'll go ahead and check those constraints.

It's also important to call out that

the query is being done to object state in the database.

Not in the instance.

That's the, sorry, that's the attributes

of the object being evaluated.

So if I'm looking at a site, for example, and applying this,

using this Backend for Object Permissions,

it's going to be looking at the database representation

of the site, not whatever might be instantiated

in memory at the moment.

Cool. So what about, so that's querying objects.

What about modifying them?

Queuering is fine if you just need to view obviously

or delete, because we know what deleting

will do to an object.

However, it doesn't address modifying objects.

So when we go to modify an object,

we need to look at both the pre modification,

so the initial state,

what it looks like before we'd done anything

and then we need to look at the post modification state.

So after we've made our changes check that it's still

in a permitted state.

So for example, if let's say I have access to,

I can change any active site,

but then I go to an active site and I change it to be

inactive or reserved, or, you know,

some other status,

the end condition then no longer is permitted.

So the, the stance that we've taken is basically

you can't modify an object

into a non-permitted state because obviously

then you would no longer be able to modify the object.

And that probably disagrees with the reason why you weren't

allowed to do that in the first place.

So how do we, how do we do that?

Right? Once our users change something,

how do we go and figure out whether the modified version

of that object agrees with the permissions?

The answer we came up with was to use Atomic Transactions.

So if you're not familiar just a real quick primer

on Atomic Transactions.

Basically we are going to wrap

an underlying database transaction into an atomic action.

So here we've invoked the context manager

with transaction dot atomic.

Then we're gonna obviously do some modify object in some way

and then call save on it.

And then we're looking for a specific exception

if something bad happens.

Then we can,

that transaction gets aborted.

So no change is being written to the database permanently.

And then we can handle whatever exception we need to.

This hasn't anything to do with permission specifically,

this is again, just a general primer on atomic transactions.

So the idea is you can make a,

you can make a temporary modification to the database,

run some other logic, and then if something went wrong,

go ahead and report that.

So here's how we're using Atomic Transactions

for permissions enforcement.

First, we're going to check the pre modified object state,

Right? So this is us using that restrict method

on the QuerySet.

So first we're going to find

the object.

So we're going to filter the QuerySet,

but using restrict by user and the desired action

and by primary key, and to get that object.

So if it doesn't exist, we're done.

There's no more going off from here.

But assuming we do find that valid object,

we move on to the second step.

So then we're going to modify it.

So let's say I changed the object status to something bad,

something I'm not allowed to do.

And then I try to save it within an Atomic Transaction.

After we call save, we then try that query again.

So we're making another database query to retrieve

the now modified state of that object.

So we're running the same query again and just checking

whether or not it exists. So it's a very lightweight query,

but basically we're saying if it no longer exists,

we know that whatever happened to that object during this,

during the previous step, altered some attributes to a state

that is not permitted by the assigned permissions.

So now I can just say, raise PermissionsViolation

and then in doing so undo the transaction.

This ensures that the object is no longer model,

it remains unmodified in the database.

We do use a custom exception here to avoid accidentally

catching it or catching

unrelated exceptions from, you know,

for other reasons.

We do have to wrap

and, you know, handle that exception.

So we have to do something with that exception once it's

been raised, ideally,

or depending on what you're doing specifically,

you might just raise and catch that. And then in turn,

raise PermissionDenied now for API views and so forth.

So other considerations to be, to be aware of,

we have to predict all avenues for object modification,

right? So this is pretty straightforward

when you're just looking at, like, the user interface

and traditional views that are serving a human.

But when you bring in something like

Django Rest Framework or graphing for Graph QL,

all those API views potentially provide an avenue for users

to modify objects. So all of those have to be,

diligently protected in the same manner.

As well, anywhere that, basically anywhere

you're calling safe, bulk update, etcetera,

any method that can modify the database has to be,

has to be controlled and secured in some way.

This approach does also work for bulk creating and editing,

by the way, we just, you know,

our examples have dealt with a single object,

but you can do it in bulk, just as well.

The only catches that reporting errors can be a little bit

tricky if you're trying to update ten things at once,

and then only like maybe two of the ten have,

are no longer valid for the, you know,

per the permissions policy.

It can be kind of tricky to figure out, okay, well,

how do I do it? Do I call out those specific two,

for example, or do I just say,

Hey, something you did went wrong.

Unfortunately right now we're doing the latter,

basically saying, Hey, you know,

one of those thousand objects that you,

that you just modified doesn't really doesn't really agree

with the policy. So try that again.

And the way in doing that is just using filter

and then passing any list of primary keys versus

a single primary key.

There is plenty of room for improvement, of course,

in the current implementation.

And that's something we're, we're working on today.

So as I mentioned,

we're working on basically taking our implementation

inside NetBox.

It's been, it was introduced back in NetBox two dot 10,

I believe. So about a year or so ago,

and it's been, it's been working pretty well.

Now we're actually starting to work

on extracting the implementation, kind of prettying it up

a bit and making it available as a standalone package.

So, you know, decoupling it from the NetBox

implementation into something that can stand on its own.

We're, it's still a work in progress.

We're shooting for end of 2021, early 2022.

And this is going to be available as a repo,

there you see the URL, Django object permissions

under NS1 Labs.

Any, you know, community help

with development and testing would be greatly appreciated.

We're pretty excited to have this split out from NetBox

and to be able to make it accessible for the, you know,

the greater Django community.

So if you're interested in doing that by all means,

you know, check it out and stop by, help us test it,

document it.

Anything else you think should be addressed or however

we can extend it to make it more usable,

more durable is obviously

of great interest to us.

So if you're interested, please check us out.

All right. With that said,

that should be wrapping up the talk.

Again my name is Jeremy Stretch.

Thank you everyone so much for attending

my virtual presentation.

I really appreciate it. I'll be available in chat

and I think a few other avenues here to answer questions,

any follow up questions, or again,

you can always just catch me on Twitter or Slack

and I'll see you around. Thanks again.