Introducing get_context_users_bycap() which gets the data in
2 DB queries, takes around 10ms on my laptop, and returns
the records with a nice context property attached.
Note Note Note: right now, some user recs do not have a context
associated, so are _not_ returned. So this awaits Matt's fix
to contexts maintenance to be 100% accurate...
A walkthrough of course-login-as functionality shows that is
Just Works, except that get_my_courses() was showing all the
courses. So we fix it.
And cleanup load_all_capabilities() - things just work
transparently.
This is *the* scary commit. No more scaffolding, no more
training wheels. Remove the legacy has_capabilty_old() and
several supporting functions.
has_capability_old()
capability_search()
is_parent_context()
roles_context_cmp()
load_user_capability()
capability_prohibits()
I've grepped far and wide. Noone else uses the supporting
functions anywhere in-the-known-universe. If I could not
find it, it does not exist.
[Thankful that cvs/git/whatever will hold on to them
if I end up having to regret this.]
This patch introduces a new global $ACCESS that will cache
accessdata (our ra/rdef array) for users that are _not_ the
logged in user.
Most of the time it will be empty (luckily).
Each new user we have to get "in" costs us 3 or 4 cheap
dbqueries at the moment, so it is not that bad.
has_capability() now calls load_user_accessdata() if the
call is for a non-logged-user. So $ACCESS is autopopulated
transparently.
This also means that has_capability no longer calls
has_capability_old(). Yay!
With this patch, user/view.php for looking at a user in a course...
Before: 62 dbqueries, 10.3MB, 0.7s
After: 49 dbqueries, 8.5MB, 0.3s
I had dropped support for multi-enrolments on the same context.
Oops!
This patch reinstates it, changing the semantics of the 'ra'
array leaves from an int to an array of ints.
So no instead of
$USER->access['ra']['/1/12/543'] = 5
we have
$USER->access['ra']['/1/12/543'] = array(5)
the functions that build the array, and the array walkers have
been updated. This touches...
Writing RAs
load_all_capabilities()
get_user_access_sitewide()
get_user_access_bycontext()
Reading RAs
has_cap_fromsess()
aggr_roles_fromsess()
Thanks to Matt Clarkson for mentioning multi-enrolments!
With this commit we now have very fast support for
roleswitching, based on $sess[rsw] entries.
And a bit of phpdoc. This is function, and the
datastructure it walks, is the heart of of it.
First stage of role_switch() rewrite - role_switch() sets up
a rsw entry in $USER->access, and makes sure we have the
appropriate role definitions in rdef.
That is where get_role_access_bycontext() comes into play -
gets all the rdef entries in one cheap sql query.
... though there may be many of them...
TODO:
- fix callers of role_switch()
- teach has_cap_fromsess() to read the rsw entries
Draft - use for the course participants list page
Uses 1 DB query (cheap too - 2~7ms).
TODO:
- implement additional where clauses
- sorting
- get course participants list to use it!
returns a users array, both sorted _and_ keyed
on id (as get_my_courses() does)
as a bonus, every user record comes with its own
personal context, as our callers need it straight away
{save 1 dbquery per user! yay!}
- Field handling moves back to get_my_courses() and now we have
almost all the fields that the old get_my_courses() did
(except for summary, which is *huge*) so get_my_courses() asks
for a lot of fields, but the get_user_courses_bycap() defaults
are _much_ leaner now.
I think this makes sense ;-)
- get_my_courses() now caches the course ids for the currently logged in
user in $USER->mycourses -- as a _string_. This is magnitudes more efficient
than having it as an array.
The cache makes a difference, but it's not very visible on
normal pageloads (with my courses block, for example).
However, over 100 iterations, for a user with 50 enrolments in a site
with 6K courses, we go from 4.3s to 0.6s. And the DB queries are *cheap*.
$tt = microtime(true);
for($n=0;$n<100;$n++) {
get_my_courses($USER->id, 'sortorder ASC');
}
error_log("took " . (microtime(true) - $tt));
get_user_courses_bycap() replaces get_courses_bycap_fromsess().
Using a combination of in-DB enrolments and in-session capability
checks, we narrow down the courses we fetch from the DB for checking.
This patch adds a small DB query, and has has a moderate impact on
the timings observable on my laptop (~300ms?), but reduces
*significantly* the bandwidth used, which in cluster environments
with frontends separate from backends has a serious impact.
get_my_courses() goes from a bazillion queries (500 in some sample
cases) to 1 for the logged-in user, and 4 for a non-logged-in user.
One of those queries brings a *lot* of data across (all rows from
mdl_course) so there is room for serious optimisation.
However, this clocks at ~300 ms on my laptop, costly, but not
the end of the world. If your PHP-DB link has bandwidth probs
it might be a problem.
A few important changes to get_my_courses()
- (Compat ALERT!) the default fields are less than before --
(will be followed by patches that fix the callers!) our defaults
had grown to quite a bit because of the crazy caching scheme it had
- the $fields parameter is to name _additional_ fields you need, and
ideally wants them passed as an array. Will cope with old-style
strings too.
- the returned courses have an extra property "context" that is
a fully valid context object, so if the caller needs to perform
further accesslib checks, we save a query per course down the road
The work is done in the newfangled get_courses_bycap_fromsess()
which is brute-force but fast. I'm sure we can optimise the common
cases a lot in it if we try. It'd be worthwhile to
- reduce the rows we grab - that's really boneheaded
- if we copy and tweak the logic of has_cap_fromsess() in it
it can be made even faster
The main thing here is that overrides in subcontexts are
now read correctly (and still cheaply) into the access array.
Also
- introducing aggr_roles_fromsess() with gives you a list of
the relevant roles that affect the user in this context
- we clear out $USER->access on login/logout
- get_user_access_bycontext() gets a few optimisations too...
Changes around load_user_capability() and has_capability() to make
the default role fallbacks and guest/nonloggedin roles work.
This commit also introduces the concept of having a magic
context next to the root context in $USER->access[ra], as
$USER->access[ra][/1] = 1 (admin roleid)
$USER->access[ra][/1:def] = 7 (loggedinuser roleid)
and has_cap_fromsess() now checks for that magic context
as well.
With this commit, the new has_capability works for the logged
in user correctly.
- load_all_capabilities() no longer calls load_user_capability()!
6K dbqueries less on login... ;-)
- When delving into a context we haven't loaded yet, we call
get_user_access_bycontext()
- Introduce: get_user_access_bycontext()
- Several fixes in get_user_access_sitewide()
(renamed from get_user_sitewide_access())
- Fixes in has_cap_fromsess()
- Introduce access_insess() to check if we have to call
get_user_access_bycontext() for the context
- has_capability() renamed has_capability_old() and we fallback
when we cannot answer the question
- has_capability() works great for the course-and-above contexts
for the logged in user - does not touch the DB
- works based on $USER->access which has a pretty compact view
of the user's access rights...
TODO:
- deal with contexts below the course - here we need to
trigger the role-assignment and role-defs load when needed
- deal with "other" contexts that hang from the system context
- deal with global roleswitch, local roleswitch
This function carefully fetches all the data needed to
handle site/category/course access for a given user
as cheaply as possible.
It currently takes 3 db queries.
Using context.path, now get_child_contexts...
- always takes 1 query
- populated the context_cache
- returns full records
- when called with an category, it won't
recurse into the children of courses
Also
- All callers in accesslib changed to the new
calling convention
A normal course page with a std blocks and a few
activities sees around 100 queries less with this patch.
Note: this commit is slightly different on HEAD/19 and on
MOODLE_18_STABLE, as groups-related tables have changed.
- On postgres query time drops from ~600ms to ~35ms in some
instances, in other instances performance is around the same
as old the implementation.
Author: Matt Clarkson <mattc@catalyst.net.nz>
