The Django tutorial is a great first resource to get
a feel for how Django works. However, Django being rather sophisticated,
I think it’s best to repeat some of those steps a few times in slightly
different settings to help internalize the knowledge.
We build here on said tutorial: for the most part, we repeat very similar
steps, encouraging the reader to use his memory over googling around, and
also progressively incorporate bits that haven’t been explored
in the tutorial.
Our goal is to progressively implement a paste service1, one step at a time.
In particular, the project is decomposed in multiple drafts, each one bringing
a set of major changes.
We thus assume that you went through the Django tutorial, and have a decent
grasp of the basics of programming. As was the case for the tutorial,
we won’t bother about deployment here (e.g. WSGI/ASGI), and rely on the
development server provided by manage.py.
The code is available on github, one tag per draft, and
generally one commit per exercise / chunk:
# Create a directory and get in% mkdir django-paste
% cd django-paste
# I'd recommend to version things with git(1). You can version# things after every exercices for example.## If you're not familiar with git(1), it's definitely the occasion# to learn how to use it.% git init
[...]# Remove some noise% echo '*__pycache__*' > .gitignore
# Create our project% django-admin startproject pastesite
# Move in and create our app (remember, a project is# a collection of apps).% cd pastesite/
% python manage.py startapp paste
# Move back to our original directory and version things.% cd -
% git add .gitignore pastesite/
% git commit -m 'creating initial project & app'[...]
First draft
Here are the features we want to provide by the end of this first draft:
an index page with a form allowing to post a new paste; the
past is a chunk of text of essentially arbitrary length;
a paste must be identified by a randomly generated string
of alphanumeric characters;
pages allowing to view a paste; the previously mentioned paste
id can be located in the URL, e.g. /p/$pasteid;
upon submission of a paste on the index page, we want to be
redirected to the page allowing us to view this paste: this allows
to communicate to the user the URL for this paste and its ID.
Exercise: I’d recommend you to stop reading here and see
how far you can implement this on your own. Try to make a conscious
effort to recall what you’ve done over heading back to the docs/code
you’ve already wrote. Even if you fail, it’ll still help commit to
things to memory.
Exercise: Why do we want pastes to be identified by a
string of alphanumeric characters? Why not a regular SQL auto-incremented
integer ID?
By default, SQL primary keys are indeed auto-incremented integers.
This means that we can easily guess links to existing pastes.
Having a sufficiently large randomly generated string offers
better guard against undesired access: the paste by default are
somewhat private.
Model
Recall that in Django’s nomenclature, the “models” correspond to Python objects,
which are “compiled” by Django to a SQL database and SQL queries allowing to access it;
said queries are wrapped by Python objects. For simplicity, we’ll work with SQLite3,
which is the default (this avoids us to have to bother with setting up a database).
Exercise: Make sure that your project is indeed configured to work
with SQLite3. You may also want to setup the project’s timezone. Finally, you
want to register our paste app within our project.
The timezone can be altered via the TIME_ZONE setting;
you might be interesting in tweaking some of the surrounding settings
as well (CET is the Central European Time):
And here's how we'd register our app; this is mandatory for otherwise,
we wouldn't be able to e.g. toy with our models from the manage.py shell.
INSTALLED_APPS = [
'paste.apps.PasteConfig', # le magic spell; others are the default'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]
Exercise: Add a Paste model. Keep is simple: it should be
exactly sufficient for our current requirements (e.g. don’t bother adding creation
dates or whatnot).
Remember that it's to be added in pastesite/paste/models.py.
This is a bit tricky, and there are different ways to achieve it; you
can, and probably should google around: it's a well-known case,
and some solutions are documented.
The trick to handle collisions is to override the save()
method: you generate the ID, then make a query via
Paste.objects.filter() to check whether that ID is already being used.
As soon as you find a free id, you're good to go!
That is the tricky part here compared to what's been done in the
Django tutorial. Everything else is pretty smooth sailing: the goal is to
force you to internalize the steps by repeating them.
Eventually, inspect the SQL corresponding to the migration:
% python manage.py sqlmigrate paste 0001BEGIN;
--
-- Create model Paste
--
CREATE TABLE "paste_paste"("id" varchar(50) NOT NULL PRIMARY KEY, "content" text NOT NULL);
COMMIT;
Execute said migration
% python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, paste, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
[...]
Exercise: You should now be able to try to add a few
paste entries by hand using the migrate.py shell. Confirm that
the database is correctly populated as well.
% sqlite3 db.sqlite3
SQLite version 3.46.0 2024-05-23 13:25:27
Enter ".help"for usage hints.
.tables
auth_group django_admin_log
auth_group_permissions django_content_type
auth_permission django_migrations
auth_user django_session
auth_user_groups paste_paste # That's the table we care aboutauth_user_user_permissions
sqlite> select * from paste_paste;
wR2J9eyLn8vcbPYs2Q48x7wu|print("hello, world!")
Admin site
Okay, now, let’s quickly recall how to setup the admin site. Let’s first register
our model so that it can be accessed within the admin panel:
# That's in pastesite/paste/admin.pyfrom django.contrib import admin
from .models import Paste
admin.site.register(Paste)
And then:
# We need to create a superuser% python manage.py createsuperuser
Username (leave blank to use 'mb'): admin
Email address: admin@admin.org
Password:
Password (again):
Superuser created successfully.
# Start the development server% python manage.py runserver
# And open a browser to http://localhost:8000/admin/# From there you can eventually create another paste.
Viewing a paste
Exercise: Create a new view which allows to
inspect the content of a paste. Remember that this implies:
Setting up the URLs in paste/urls.py and in pastesite/urls.py,
as recommended in the official tutorial;
Probably setting up a template in paste/templates/paste/paste.html
(remember that the convoluted pathname is because we want to namespace
our templates as they could be accessed from elsewhere);
Okay, let's start with the URLs and ``paste/urls.py``
from django.urls import path
from . import views
app_name ='paste'urlpatterns = [
path('p/<str:id>/', views.paste, name='paste'),
]
This needs to be registered in ``pastesite/urls.py``:
from django.contrib import admin
from django.urls import include, path
urlpatterns = [
path('admin/', admin.site.urls),
path('', include('paste.urls')),
]
There's not much more difficulty regarding the view creation;
I'll leave the template out to save space.
from django.shortcuts import render
from django.shortcuts import render, get_object_or_404
from .models import Paste
# Create your views here.defpaste(request, id):
p = get_object_or_404(Paste, pk=id)
c = {"id" : p.id, "content" : p.content }
return render(request, "paste/paste.html", c)
Submitting a paste
Exercise: Create a new “index” view containing a form,
allowing to submit a paste. Upon submission, the user should be automatically
redirected to the corresponding paste viewing page. Again this implies
setting up urls, templates, and writing the views.
And the views are dead-simple; again I'll leave the templates
out to save space:
from django.shortcuts import render, get_object_or_404
from .models import Paste
from django.urls import reverse
from django.http import HttpResponseRedirect
defpaste(request, id):
p = get_object_or_404(Paste, pk=id)
c = {"id" : p.id, "content" : p.content }
return render(request, "paste/paste.html", c)
defindex(request):
return render(request, "paste/index.html")
defnew(request):
p = Paste(content=request.POST["content"])
p.save()
return HttpResponseRedirect(reverse("paste:paste", args=(p.id,)))
Second draft: adding dates, preparing the third draft
Adding dates
Let’s start with something that hasn’t been done in the Django
tutorial: we have a database, a table to hold our pastes, and we
now want to extend our Paste model to include a few new
fields: in particular we want to store:
a creation date;
and, in preparation for the next draft, a modification date.
Exercise: How would you naively tackle this? What issue
in particular are you expecting? Do you think Django will be able
to help? If so, then how? Think about it a little and then give
it a shot.
Well, we kinda hav been sold on the idea of "migrations": supposedly,
django will be able to compile model changes to a migration script, so
it remains to see if django holds up to its promises.
Let's give this migration process a try. However, one thing
which might cause us trouble is that, if we add new fields, and
if our database isn't empty, we'll need to decide what value to
add to those fields.
Now, in a production setting, depending on the cases, perhaps we'd
want to compute a list of values from an auxiliary sources (e.g.
HTTP logs), and adjust the migration script accordingly. But in our
case, we can just listen to Django: after having added the two
following fields to our Paste model:
Django complains as such (remember, migrations are performed in
2.5 steps: compute it, inspect it eventually, apply it):
% python manage.py makemigrations paste
It is impossible to add a non-nullable field 'cdate' to paste without specifying a default. This is because the database needs something to populate existing rows.
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows with a null value for this column) 2) Quit and manually define a default value in models.py.
Select an option:
Let's for example choose the second option, as this is quite reasonable;
note that we have to work a little to shut a few warnings; observe also
how we can access the configurations settings set in pastesite/settings.py:
And from there we can compute, observe and apply our migration as usual:
% python manage.py makemigrations paste
Migrations for'paste':
paste/migrations/0002_paste_cdate_paste_mdate.py
- Add field cdate to paste
- Add field mdate to paste
% python manage.py sqlmigrate paste 0002BEGIN;
--
-- Add field cdate to paste
--
CREATE TABLE "new__paste_paste"("cdate" datetime NOT NULL, "id" varchar(50) NOT NULL PRIMARY KEY, "content" text NOT NULL);
INSERT INTO "new__paste_paste"("id", "content", "cdate") SELECT "id", "content", '2024-08-13 20:00:51.533453' FROM "paste_paste";
DROP TABLE "paste_paste";
ALTER TABLE "new__paste_paste" RENAME TO "paste_paste";
--
-- Add field mdate to paste
--
CREATE TABLE "new__paste_paste"("id" varchar(50) NOT NULL PRIMARY KEY, "content" text NOT NULL, "cdate" datetime NOT NULL, "mdate" datetime NOT NULL);
INSERT INTO "new__paste_paste"("id", "content", "cdate", "mdate") SELECT "id", "content", "cdate", '2024-08-13 20:00:51.536116' FROM "paste_paste";
DROP TABLE "paste_paste";
ALTER TABLE "new__paste_paste" RENAME TO "paste_paste";
COMMIT;
% python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, paste, sessions
Running migrations:
Applying paste.0002_paste_cdate_paste_mdate... OK
Exercise: Adjust the paste showing view to display both dates.
This is really easy, you just have to feed two additional values in
your template and display them.
Tweaking migrations by hand
If you look at the SQL code generated by the previous migration, you’ll
see that each column is added one at a time. Let’s pretend that we think
it’s a bad thing (e.g. “it’s too slow”) and that we want to
tweak that migration to perform both stages at once.
Note: This is “for fun”: the intent is to give you some practice
for the day you’ll have to deal with things like this in a more serious
setting…
Exercise: Revert our last migration. Try to see if you can
replay it. Make sure the migration is reverted before moving on
(meaning, if you’ve replayed it, revert it once more).
By default, you shouldn't be able to replay the migration "easily":
that's because the migration is still registered as being applied
by django: you need to figure out where this registration occur
(try poking around the SQL database...)
You should notice that one "can't" just run the migration again:
% python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, paste, sessions
Running migrations:
No migrations to apply.
If you've versionned the db.sqlite3, the file containing
the database, you can checkout what was there before unrolling the
migration (e.g. git checkout). An even better approach
is to make the migration as being unapplied: Django migrations are
registered in the SQL database in a special table, django_migrations:
As you can see, the fact that the applied field
is characterized as NOT NULL seems to hint that
removing the entry corresponding to our migration from the
database should do the trick. Let's try:
-- Be careful: the id might be different for you:
sqlite>deletefrom django_migrations where id =20;
We can then check that there's now indeed a migration to
be applied, and re-apply it:
% python manage.py migrate --check || echo a migration is waiting
a migration is waiting
% python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, paste, sessions
Running migrations:
Applying paste.0002_paste_cdate_paste_mdate... OK
% python manage.py migrate --check && echo no migration to apply anymore
no migration to apply anymore
[0m% python manage.py dbshell
SQLite version3.46.02024-05-2313:25:27Enter ".help"forusage hints.
sqlite>select*from django_migrations;
[...]
18|paste|0001_initial|2024-08-1308:04:22.34659919|sessions|0001_initial|2024-08-1308:04:22.62754621|paste|0002_paste_cdate_paste_mdate|2024-08-1502:31:34.812885sqlite>deletefrom django_migrations where id =21;
Exercise: Now, grab/compute the SQL migration code so as to
add the two fields in one fell swoop; execute the code, and find
a way to “trick” django into thinking that the migration has been
successfully applied.
This means we want to have some notion of ownership, but we’d like
to keep the service essentially anonymous. A quick way to authentify
a user would be to provide the paste creator with a unique “secret”
token (to fix the ideas, make it alphanumeric bytes long), which
he could later use to justify of his identity.
Of course this isn’t flawless: for example, were the user to lose
that token, he’d lose all privileges over the paste. Nevertheless,
it’s a reasonable engineering solution for a paste service.
Exercise: Without reading much further, think about it
a little, and try to implement those features, in the way that we’ve
just discussed, on your own.
We’ll propose below a way to implement this one bit at a time.
Adding a new field; manual update tweaking v2
Exercise: So, we probably should start by adding a new
token field to our Paste model; do so, and look at the
code of the corresponding SQL migration: what do you see?
This is rather straightforward: let's add the new field for example
as such:
defmkrandstr(n : int) -> str:
return''.join(random.choices(string.ascii_letters + string.digits, k=n))
# remember, we can't use a lambda in "default" because they're not serializabledefmkrandstr32() -> str:
return mkrandstr(32)
[...]
classPaste(models.Model):
[...]
token = models.CharField("ownership token", default=mkrandstr32, max_length=32)
[...]
Let's then create the migration, and compute the coresponding SQL:
% python manage.py makemigrations paste
Migrations for'paste':
paste/migrations/0003_paste_token.py
- Add field token to paste
# Again, re-indented for clarity% python manage.py sqlmigrate paste 0003BEGIN;
--
-- Add field token to paste
--
CREATE TABLE "new__paste_paste"("token" varchar(32) NOT NULL,
"id" varchar(50) NOT NULL PRIMARY KEY,
"content" text NOT NULL,
"cdate" datetime NOT NULL,
"mdate" datetime NOT NULL
);
INSERT INTO "new__paste_paste"("id", "content", "cdate", "mdate", "token") SELECT
"id", "content", "cdate", "mdate", '9EFAthChoZxETG5NhLWnBixWGqI6MXs6'FROM
"paste_paste";
DROP TABLE "paste_paste";
ALTER TABLE "new__paste_paste" RENAME TO "paste_paste";
COMMIT;
Now, an "issue" is that all existing paste will share the same
ownership token. It's not really critical, especially since we're
not in production or anything, but for practice, we'll see in the
next exercise how this can be tweaked.
Exercise: Try to find a way to solve the problem mentioned in
the previous solution. Again, it’s not really crucial, but it’s good
practice to get a feel for how much we can bend Django migrations.
You can think of this approach as a refinement over the one we took
in the previous draft.
The trick is to edit the migration files previously created by
python manage.py makemigrations paste. In particular,
have a look at the documentation of
migrations.RunPython()
The idea is to add an extra step to our migration's operations,
which will iterate over all existing pastes, and manually update their token
fields; in paste/migrations/0003_paste_token.py:
[...]
defsetdeftoken(apps, schema):
ps = apps.get_model('paste', 'Paste')
for p in ps.objects.all().iterator():
p.token = paste.models.mkrandstr32()
p.save()
# Nothing particular to be done heredefreverse_setdeftoken():
passclassMigration(migrations.Migration):
[...]
operations = [
migrations.AddField(
model_name='paste',
name='token',
field=models.CharField(default=paste.models.mkrandstr32, max_length=32, verbose_name='ownership token'),
),
# This is the new step we're adding migrations.RunPython(setdeftoken, reverse_setdeftoken),
]
We can then apply the migration:
% python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, paste, sessions
Running migrations:
Applying paste.0003_paste_token... OK
And check that our token fields are in general distinct:
% python manage.py dbshell
SQLite version 3.46.0 2024-05-23 13:25:27
Enter ".help"for usage hints.
sqlite> select content, token from paste_paste;
print("hello, world!")|VpIhzzPqnRXsdXKFhPV8qjkEEZuQ8sod
echo "hello world!"|nE5T46xCqWzwt6QtZcqIKAMUlWr2jzOg
“Sending” the token to the paste owner
Exercise: For practice, let’s first implent this in a clumsy way:
once the paste is created in the new() view, we perform a redirect
to the paste page; add to this redirection a GET parameter
containing the token. What’s the problem with this approach?
defnew(request):
p = Paste(content=request.POST["content"])
p.save()
return HttpResponseRedirect(reverse("paste:paste", args=(p.id,))+"?token=%s"% p.token)
Note that the GET parameter name is arbitrary, as it's never used
elsewhere, at least for now.
Now, what's the issue with this method? Well, it's an UI issue: the paste
creator must know that there's an ownership token located in the URL,
and mustn't forget to remove it from the URL before sharing it.
Exercise: Alright, let’s see if we can do better: essentially, what we
try to achieve is have data flow from the new() view to
paste(). We could for example store that data somewhere, on the
server in new(), and check later in paste() if there’s
anything for us.
This is typically implemented with something called “sessions”, of which django
provides us with a ready-made implementation. The idea behind sessions is that
django will keep some data associated to an user somewhere on the server side, and
send that user’s browser a cookie. The user will then send back that cookie at
every request: the server can then use that cookie to pick up the data associated
to the user.
So, try to use sessions to store the token and send it back to the user. The token
must be provided in a new form located in the paste() view, pre-filled
with that token, at least upon paste creation. In the next section, we’ll add
buttons to that form to edit/delete the paste.
By default, the sessions will be stored in the DB; there are other
approaches (e.g. files, memory), but the DB will be good enough for us.
Then, it's just a matter of storing/retrieving data from/to
request.session:
Note that by having one session entry per paste created, we naturally
don't pollute random pastes with the token for another paste, and
each paste owned by the user will have that field correctly pre-filled,
at least for as long as the session last.
The template bits are really trivial, for example:
Regarding the error, because it'll be triggered in the view(s)
handling the actions, and because that view will likely be reached
by POST, and will "have to" redirect to another view,
you'll probably want to store the error in the session.
There are two interesting bits: the hidden field containing the
paste id, and two buttons, tied together by the same name attribute,
but containing different values corresponding to our two actions.
We can then have a single route do/ for both methods.
Here's our route registration in paste/urls.py
Note that we could have stored the paste id in the form's URL,
and retrieved it via the route with do/< str:id>,
which would have required a view def do(request, id).
Now, potential error is stored in the session, so
that it can be retrieved once we redirect the user to the page
corresponding to the paste we've tried to act upon, after the POST
request. Here's how this error is eventually caught in the paste view:
Comments
By email, at mathieu.bivert chez: