If you want a batteries-included, zero-config Django + React framework, check out https://www.reactivated.io .
It incorporates most of these best practices, along with React server side rendering in addition to regular Django templates.
Full disclosure: I'm the creator.
If you do this a lot you should look into ‘startapp —-template’ which lets you bake in boilerplate like this.
https://docs.djangoproject.com/en/4.0/ref/django-admin/#cmdo...
One minor point on app names. They are really hard/annoying to change once you have production data, because your DB tables will always be appname_modelname, and renaming tables is a real PITA. You can set the model to point to a nonstandard table, but then your project has a confusing non-Djangonic wart that will annoy you forever more. For this reason I think ‘core’ is a safer option, unless you are certain what your app should be named. Naming after one domain model is likely to be too specific. (I strongly recommend against multiple apps until you have functionality you need to commonize between projects, as refactoring models between apps is a nightmare if you get the boundaries wrong.)
Regarding apps, I was just thinking you could create a directory structure like this, essentially eschewing the whole concept of Django "apps":
<project>
src/
<package>/
models/
__init__.py # Import all model classes (necessary for Django to find them)
some_model.py # Define SomeModel class
views/
__init__.py
some_view.py # Views for SomeModel
settings.py # Add "<package>" to INSTALLED_APPS
urls.py
If it turns out later you need resusable "apps" for some reason, you could always add an apps/ subpackage for them.
I haven't tried this in a real project, so I don't know if there are any downsides, but it seems like a decent approach.
Yeah I've played around with this approach in the past while prototyping service boilerplates, I think it's viable. I never got it polished enough to publish as a startproject --template and ultimately went with Flask for microservices, so I didn't finish the prototype.
I found this article useful while I was experimenting: https://simpleisbetterthancomplex.com/article/2017/08/07/a-m...
There are a few different places where little things break and then you need to use an obscure settings.py variable to fix them, which makes me a little nervous since it will cause a little cognitive friction for Django-fluent developers joining your project. But I do think it's worth experimenting with.
> For this reason I think ‘core’ is a safer option, unless you are certain what your app should be named.
Yep, this is what I do. It’s rare that I make anything that can be shared cross-app (most Django apps I make are super simple CRUD-types), so every one of my Django apps has a ‘core’. I also recommend this approach.
I would add to this list: convert your template engine to Jinja2. I held out for a long time on this, and now I would not go back. The purist approach ("keep logic out of templates") sounds good, but in practice leads to spaghetti hacks with weird additional template tags to do simple things, a bunch of additional context variables with no meaning, etc.
> purist approach ("keep logic out of templates") sounds good, but in practice leads to spaghetti hacks
This. Exactly this
Go for Jinja2 and don't look back
If there's one defect in Django is the rosy-colored purist view that permeates a lot of its design decisions.
Keeping "logic only in code" made sense in the 90s. And no, I'm not building a new template tag every time I make some stylistic choice.
About the purist view on Django: The django templates don't annoy me that much, but what is really annoying is the avoidance of using thread locals in Django. This makes very difficult stuff that would be trivial in other frameworks (ie flask), like getting the current user.
As another hold-out on Jinja, what pushed you over the edge? Half of the appeal to me in Django is that if you follow the easy path, one project should look like another. Breaking conventions for a few template niceties has not seemed worth it (although, I am not a front end guy, and survive on very barebones presentation).
> what pushed you over the edge?
Using Tailwind without a frontend js framework. I needed a way to create lightweight “components” ala React but on the backend, because otherwise you’re either copy pasting classes like mad or (ab)using tailwind’s @apply.
I had used this project for creating components https://github.com/EmilStenstrom/django-components/
Also, for simpler cases I include partial templates.
How do you create your components in Jinja2? Using the include tag?
I've been gradually removing all use of Jinja macros from our projects as they make debugging so much more painful. Everything now goes in template globals, which lets you debug them the same as anything else, gives far clearer stack traces and lets you do clever things with caching.
Straightforward, useful advice. I pretty much do all of these when I start new Django projects.
The need for a custom User model makes me a little sad every time because, well, single `name` fields (rather than `first_name`, `last_name`) and email-for-username do feel to me like more sensible defaults circa 2022.
I'm curious as to why you think a single name field is better than the traditional separate first and last?
Per https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... , names have far too much ambiguity to consider that two fields is necessary or sufficient to encapsulate that information.
Maybe for consumer apps, but for a lot of enterprise apps that have to be compatible with existing APIs or domain-specific data standard that require two name fields, that’s not really a viable approach.
I’ve never really understood the concept of an ‘app’, in some projects I’m sure in makes sense but it not being transparent by default in a new project seems silly.
Am I missing something here?
I used to build everything with one giant “monoapp” and completely ignore that aspect of Django, which worked pretty well. Eventually though you need to split up your models into multiple files, and then your views, and admin, and so on, and it turns out there’s a lot of benefits of just segregating things into “apps” from the beginning. The only real downside I’ve noticed in practice is the omni-file-search feature in my IDE is harder to use, because you now have 7 “models.py” and it’s a bit harder to quickly navigate to the right one. If you split up files without using apps (just normal Python imports), you don't have the same problem with all the names being duplicated.
The pitch that you can "reuse" apps across projects always seemed weird to me, because that's basically never something you actually want to do/can do easily without major surgery.
Protip on separating your models etc out: use modules instead of apps. so instead of a models.py have a models folder with a structure like this
- models
cheese.py
spam.py
__init__.py
The same trick of course applies to any python file you want to split up. Views, urls, managers, etc etc.
Yeah that works pretty well too. I think there’s some minor advantages of having them in apps, they’re sorted neatly in the admin, you’re forced to split views, models, templates, admin, etc together which keeps things a bit cleaner. But it’s also messier in other ways (you have to sift through too many files to find the thing you want).
Migrations that cross app boundaries are a nightmare. Before you know it there's thousands of migrations in your various apps, and squash is powerless to help you. It'll pretend to work but break your migrations if you try to squash across any migration that depends on a migration in a different app.
IMO migrations in combinations with apps are fully broken in django. it's unworkable for large projects.
Shameless plug, I've given a talk about scaling Django codebases to many apps at PyCon (2021) and PyCon UK (2017). At my previous job we had around 500 apps in our codebase and it was honestly great to work with.
Here's the link to 2017 talk: https://www.youtube.com/watch?v=jBBcORHhfV0 Couldn't find 2021 talk that fast..
Thanks, here's the 2021 talk.
I will sign onto this approach as well. Never expanded beyond a `core` app, which may have duplicated modules as the project grows (models_foo, models_bar, views_foo, views_bar, etc). I have never felt that any one component of the project was isolated enough from the rest of everything that it made sense to firewall off pieces into apps.
I’ve never successfully isolated them in the sense of having no cross imports, but I think there are advantages keeping the code broken up anyway.
Say you want to easily create a comments app or ratings that you can associate with any other model in your project. Write that app once and use in many places. Or maybe something that sucks to write that may be error-prone, like auth, especially social auth which may change frequently. Maintaining one app and sharing it is way better than everyone trying to keep up with the changes themselves.
Or take a look at https://djangopackages.org/ for examples of apps people like to share.
Can't you just keep the logic for your comments app in the monoapp and use it with everything?
I understand if you want to distribute your app for others to incorporate, but otherwise it felt like the documentation overemphasizes the "app" aspect. It's a reasonable structure but more of a guideline... unless you want to reuse in other codebases
I think many (most?) Django apps only ever need one app. I think the fact that it's front and center in Django is a bit if a distraction that makes people think they _have_ to use multiple apps.
100% agree. It’s a distraction
The name suck, but the idea is they are plugguable: check the django packages site for an idea of the benefit of that. It's essentially a plugin system with autoloading of some resources, such as the model and templates.
Also, apps override each other in the order or import. E.g one app can override the template of another just by using the same name. This means you can plug AND extend.
And also install Django Extensions (https://django-extensions.readthedocs.io/en/latest/)
It makes the Django CLI comparable to rails. Notably the reset_db and shell_plus command
shell_plus is fantastic.
Reasonable. Personally I just made a template project to get everything off the ground in one step, even the production environment and repository.
I don't know why no one mentioned this.
https://cookiecutter-django.readthedocs.io
I use a modified version of this for all my projects. This one comes with all the goodies
Yup, a custom cookiecutter template saves ton of time.
I created this one to create a production ready Django project in few minutes: https://github.com/Mittal-Analytics/django-cookiecutter
Things this does:
- README.md: setup readme with development setup
- Django split settings: split settings for local, testing and production
- Split requirements: split requirements.txt for local and production
- Pre-commit hooks: setup pre-commit hooks for black and pyflakes
- django-envoiron: database config and secrets in environment
- editorconfig: sensible tab/space defaults for html, js and python files
- remote-setup: setup hosting on uberspace
- git push deployment: `git push live` makes the changes live
- github actions for tests: run tests automatically on GithubReally sensible, straightforward recommendations. Nothing flashy.
Honestly I think these should be defaults (or at least configurable options) of the project generator. I guess you could use cookiecutter, but it's not really worth it.
This depends. If you start a new project often it is much better to use a template.
7. completely rewrite the auth system because it's not sufficiently flexible, chafe at the boilerplate, then port the backend to a different framework + miss the admin system a little
Django-allauth usually got you covered, but in the rare case it doesn't, creating a micro service just for auth is a way better alternative than a complete rewrite.
painfully true
I do pretty much the same, which makes me think it should be django default startup template.
I would also do:
- install django_extensions and django_debug_toolbar
- rename the admin URL route to contain a uuid
- setup black, mypy, pylint, ipdb, ipython, dotenv and doit (I do so for all my python projects)
- manage.py createsuperuser
Also, very often :
- setup celery (locally with fs backend)
- setup vitejs
- install sentry and mailtrap plugins
- install django-allauth
- install DRF
And currently exploring:
- replace DRF with django ninja
Using a boilerplate template can really speed things up, and more importantly, avoid forgetting to set up (or change from default) something that's hard to change later, like the user model.
I do a lot of Django+DRF projects and built https://apibakery.com/ to help with the initial scaffolding and defining the models (started as my internal tool, later opened it up to the public).
One thing I never figured out was dependency management. Some others are linking to two scoops of django and other starter templates, it seems like none of them get past a kludgy set or requirements.txt files. I want something easy to use like ruby bundler or npm. I want to be able to specify my dependencies and install them in one command, and specify if something is production or local in that same command.
Have you tried Poetry? (https://python-poetry.org)
Out of curiosity, and because I rather prefer pip and the requirements.txt file over npm or anything Ruby has to offer, what issues do you encounter with dependency management in Django/Python.
Granted, trying to build your own reusable modules for Django is not a great experience. In these cases the tools feels rather restrictive.
> I want to be able to specify my dependencies and install them in one command
"pip install -r requirements.txt"
If you have different requirements, you can use a different file.
Obviously this is only for development, and if you package your Python code properly then you can install the whole damn thing as a package.
I use pip-tools with venvs for that and can‘t complain. It pins dependencies and you can create different requirements.txts for local dev and production (production for me is usually dev minus some debugging tools). It‘s not particularly fancy, but it gets the job done.
And a way to manage the update of them… currently doing this blindly, running the test suite after each updated package hoping nothings broken.
Slow and frustration prone with “sorry this package require packagex version 1 but packagew require it to be 2 now”
I used Django, and personally I don't like Django startaproject command that not allow us to create project that contains dash "-". Also default appname has the same name with the project name can confusing the new comer in Django.
I always structures my project like this when start my Django project.
$ mkdir -p my-example-project/src
$ cd my-example-project/src
$ django-admin startproject youtube_downloader .
so all Django related will be under src folder.
That's actually a Python convention issue rather than a Django issue. Python convention dictates preference for underscores. This is because dashes cannot be used in a module name.
Yes, because default Django app name by default using the same name with Django project name.
Every time whenever I start a new project.
1. django-environ 2. celery 3. channels 4. apps structure https://rajasimon.io/blog/django-project-structure/
To setup all the "Hello World" take long time.
How often does one start a new Django project? I've been working on the same project for a couple years...
5 times a year if you are a freelancer.
I started dozens of rails projects when doing client work. We occasionally chose Django, too.
The first two can be covered by this helpful library. [0]
Why don't you make a project template instead and start from there?
That's what I do, complete will all the dependencies I always need (like Celery, Allauth, etc)
Why not ... expressJS + typescript + React then ? Not sure what's the point of using Django, except for You only know Python reason
Really irrelevant comment. Moreover, the "You only know Python" reason is a very strong argument.
It is almost like one could leverage python and all the amazing libraries to compute interesting things.
It is almost like we can actually build web apps that do more than CRUD.
Short answer: Django comes with so much out of the box that you end up reinventing a lot of wheels if you go with a pure JavaScript/TypeScript approach.
all great bits of advice except the first.
keep your secrets out of your environment! especially when you aren't providing any additional safety advice like making sure DEBUG is off
Some reasons:
- https://diogomonica.com/2017/03/27/why-you-shouldnt-use-env-... - https://blog.forcesunseen.com/stop-storing-secrets-in-enviro...
They both recommend mounting them into a tmpfs, and then unmounting, but I've not yet tried this.
Environment variables exist only in memory and can't be read by any user other than the process's own and root. There's nothing wrong with keeping secrets there.
Depends of the size of your setup. Most website are small, and their threat model don't require more than having the secrets in a systemd file env statement.
After all, if you have a single server, and the attacker can read a root protected file oe the spawned processes context, you are pwned already. As for exposing env var, popping os.environ is usually enough.
No need to pay for more than you must: bots and script kiddies are not mossad.
I've heard this advice a number of times, but often run up against otherwise standard looking systems that rely on secrets in environment variables -- mainly thinking about AWS's requirement when using Secrets Manager with ECS; the secrets are stored securely, but ultimately loaded into a containers environment.
Where should you put secrets instead? Obviously not the filesystem.
> Obviously not the filesystem.
How would you avoid that?
Even this article, which recommends keeping your secrets in environment variables, tells you to implement that by storing them in the filesystem. The advice isn't to avoid storing your secrets in the filesystem; it's to avoid storing them in version control.
It is apparently an exercise for the reader to figure out why it's better to read your secrets out of the environment, which read them from a file you provided, than to read them from your own file yourself.
Linux already stores plenty of secrets in files in /etc. Just do the same: the root protected init file for your app likely has a mechanism for passing an env var to the new process. Systemd and supervisor do. Then pop os.environ instead of reading it, and you are safe from bots and script kiddies, which is likely your threat model
If you use bitwarden - you can set them via bitwarden cli, eg:
export DATABASE_URL="$(bw get password my_app/db_url/prod)"
This has the same problem that OP refers to with regards to DEBUG: It remains in the environ, and for example if you forgot to set DEBUG=False, Django will just dump the whole environ on the error page.
If you can call bitwarden from within the python code somehow, such as with subprocess, that would be better.
This is only a solution for inputting the password. Generally you would set it as a variable on the server (eg: heroku sets this) - but you could provide it on startup for local development/testing.
And what happens on reboot ? Then you get to hardware solutions, and a small website is not going to want that.
Env vars in systemd or supervisor files are fine for small server projects. But make sure you pop them from the os.environ dict once you read them to avoid accidental exposure.
For the desktop, use the keyring module.
If you start to scale up your threat model, you should use a vault, but the setup is way more costly, and tricky to get the reboot story right (hence the priviledged first requests comments).
> But make sure you pop them from the os.environ dict once you read them to avoid accidental exposure.
I've always accepted this was an attack vector, and some malicious library could extract env vars.
This actually makes a lot of sense, thanks!
Theoretically you could write a process that gets its secret from the first request, which is treated as privileged. Then the app can have it in memory, and it's only as vulnerable as the app itself. (This is a kind of degenerate form of the "stem cell" pattern for processes, like Erlang's gen_server.) And yes, this kicks the can to another system, but presumably the secret has to be durable somewhere. Although perhaps a sufficiently sophisticated (or broken?) cluster the nodes could keep passing the secret to each other, never touching disk.
I’ve heard that tmpfs is a good solution. A script on boot loads the secrets from something like vault. Env can be exposed through docker inspect if you’re on a shared host. Not sure what other negatives to env there are though.
Things I do every time I start a Django project:
1) bring up Django project
2) populate db with example data
3) "Wow auto admin is so cool!
4) Right anyway, let's get to work
5) mkdir my_real_project && cd !! && pip install flask
The first thing I do when I start a Django project is to not use Django.
Which is good. Imagine trying to scale a Django app when more than just 5 concurrent users are hitting it. Step 1, don't use Python.
it’s not that bad
What else do you use then?
Good advice
I found this a lot more interesting than the submission article: https://12factor.net/
Tangent comment on JSX/TS/JS and a snippet found on reactivated.io. Not a comment on the article or reactivated.io in any way.
> export const Flavor = () => <Select options={flavors} />;
When I read the above, I fear for the new devs. The number of concepts to grasps and the level of parsing and mind bending to do is incredible.
I used to teach code, before arrow functions and React. And I am convinced I would have lost some students to this snippet, especially the "= () =>" segment.
Are we gone too far syntactically for the sake of compactness rather than understandability, allin the name of (fake, IMO) readability?
Some of that shifts with syntax highlighting: drawing the eyes to which bits are more specifically related. The arrow => should highlight as a single operator (just as >= does), for instance. Most people call these "arrow functions" so thinking of "=>" as the arrow operator is a habit that quickly builds.
They are controversial and everyone has different opinions, but I also think this is where programming ligatures come in extremely handy. When => looks more like ⇒ and is even more obviously an arrow, I think that also starts to make it easier to visually "parse" the flow of code like that.
For what it is worth, arrow functions weren't added solely for compactness, but also to fix some historic issues with classic function syntax. (Lexically scoped `this` versus caller scoped `this` being the big one.) A new type of function syntax was desired anyway for those reasons, and the compact syntax was the icing on the cake.
I haven't had the pleasure of teaching such things to students at this point in my career, but I have given a hand to many a junior developer to grasping some of these ideas and I don't think it is that tough, though it can be a shock/surprise if the last JS you touched was many years before. Especially if you are trying to also learn what JSX and/or TSX add on top of all the changes in ES2015+.
No. That's perfectly normal syntax you would use every day. It is not sophisticated or special.
Also, consider functional languages like Haskell or whatever. Their syntax is exceptionally foreign for users of C-like languages, and so are the concepts. Still, they're (relatively) widely used.
Comparing something to something else harder does not move the former into the realm of easy things.
Haskell though requires significantly more work than JS to grasp, for sure.
If you work with JSX daily, which 99% of React developers do, this is not hard to grok. It's a very typical functional composition pattern in React. Since this framework is targeted at those who want to use React, and presumably already use it, I don't see the issue. It's a snippet to show how Reactivated lets you use the React ecosystem, not how to use React - that's what the docs are for.
> Are we gone too far syntactically for the sake of compactness rather than understandability, allin the name of (fake, IMO) readability?
One of the reasons perl faded away, imho
I’m not sure that brand new devs who have trouble reading standard JSX, are exactly the target market for a bleeding edge project that combines two popular frameworks together.
I think the target market is experienced people who want to use both react and Django, and feel that this project saves time on conceiving and writing their own integration.
`() =>` is just the lambda syntax. JavaScript has, by far, one of the most understandable lambda syntaxes.
For instance, look how blocks are declared in Objective-C: http://fuckingblocksyntax.com/
Compactness and understandability are the same thing on a large scale or under time pressure. This example is definitely loaded, but compared to more wordy alternatives it just cuts to the chase.
> Compactness and understandability are the same thing on a large scale
Could you elaborate on that, please? The way I read this, I could not be further from agreement :)
I appreciate your ~writing~ coding style ;)
Maybe we can find consensus by avoiding the extremes.
"Extended code" as you name it is certainly bad with regards to explicitness, while extreme compactness like (some) one liners are as bad as the former but with regards to readability.
Both are negatively impacting understandability.
If you're doing server-side rendering, why do it in React SSR instead of Django templates?
I've seen this pattern a lot recently but haven't figured out what the extra complexity gets you
I haven't used this particular framework, but having type safety in React through writing TypeScript, generating bindings to the backend so those are typed as well.
There's also a _ton_ of preexisting React components you can generally drop in and use, which is less true these days with something like a Django template.
You also have the option of doing SSR and then doing more dynamic stuff on the client side, as a sort of optimization (and plain better user experience than making _more_ server requests to get the initial page state loaded).
Thanks for taking the time to reply, but I'm still not really understanding the benefit.
Type safety is incredibly important if you're lots of logic in a language, which is why TS is great for SPAs. Does the type safety of TS get you anything if you're just doing SSR with not a whole lot of logic in TS? All of your application logic will be on the Python side of things.
>You also have the option of doing SSR and then doing more dynamic stuff on the client side
Isn't this just the old-school way of doing things before SPAs came around? i.e you render the page on the server and then add dynamics features using JS. I think the new way of doing this is with htmx, hotwire, etc.
I think the biggest benefit of SSR is you get the first page fully rendered out (good for SEO), and beyond that page the react SPA takes over by doing all the cool client side stuff like routing and what-not.
The biggest benefit for SSR in my opinion is SEO + first load is fast because its already generated. After that though, its just the plain old SPA experience.
It's because React (and other SPA technologies that also happen to work with SSR) is all the buzz. It doesn't actually necessarily make sense. The risk to a project is usually NOT the technology chosen for frontend.
Django templates are perfectly fine as long as you leverage template tags the way they were intended.
I would agree that "Django templates are perfectly fine"... until: https://www.reactivated.io/documentation/philosophy-goals/#u...
Of course there's many ways around this, but doing them in a declarative, type-safe way is not trivial when using Django templates.
It's very similar, except hotwire/htmx won't give you server rendering. It's not that big a deal if it's just small snippets.
This does make testing a lot easier, as the server response is identical to the hydrated client.
There's a lot of good advice in that article. I liked the part on cookies in particular. You rarely read that anywhere.
You can use the same templates on the backend and frontend. That may not matter to you, at least not now, but it's a great reason in many contexts.
Personally I find JSX to be a terrific templating system, much better than anything like Django's, but if you're familiar with both and prefer Django's, this benefit does not apply to you!
Oh wow, this looks really good. I want to say, what I was really hoping to see is a "how does this work?" section. I'll read the source code, but it would be nice to have a quick narrative explanation.
EDIT: Looks like the "Concepts" page has what I am looking for. I would add some of that to the front page.
this is cool, thanks for sharing. this (https://www.reactivated.io/documentation/philosophy-goals/) fits my bias so i'm def gonna try it. curious how it works at a low level, too
Same, I agree with many of the thoughts there. When you get a moment, would you mind writing a guide on how to deploy it to Render.com? It sounds like the Heroku of 2022.
Theoretically, the Dockerfile should "just work"™ with Render.com as well. Right now I focus on fly.io only because their free tier offers PostgreSQL without time limits. Render, I believe, only does so for a period of time.
See here: https://github.com/silviogutierrez/reactivated/blob/main/dev...
https://github.com/silviogutierrez/reactivated/blob/main/dev...
https://github.com/silviogutierrez/reactivated/blob/main/dev...
Can you link any sample projects that currently uses reactivated?
Sure:
The docs site itself: www.reactivated.io . Notice no JS loaded on the client side. It's purely SSR. The code is in the repo under /website.
My personal blog: www.silv.io
And my business: www.joyapp.com
I don't track others that use it, but hopefully they're out there!
Great thanks.
This looks great!