Implements a status check which confirms that the Solr search engine
is available. Optionally, the check can also show a warning if the
index grows beyond a certain size.
As part of this change, a new API was added in search_solr\engine
to allow using http_client (Guzzle) instead of raw Curl; this makes
it easier to create mock tests in PHPunit for the new functionality.
All setUp(), tearDown(), setUpBeforeClass() and tearDownAfterClass()
must, always, call to parent, to ensure that everything is properly
set and cleaned.
While in a lot of situations this is not needed (parents may not
have anything to run), with PHPUnit >= 10 this can become more
important because we are going to move the reset code from current
placement @ runBare() to setUp()/tearDown().
Note that all the changes performed in this commit have been detected
and fixed by moodle-cs (ParentSetUpTearDownSniffTest).
While this change is not 100% required now, it's good habit
and we are checking for it since Moodle 4.4.
All the changes in this commit have been applied automatically
using the moodle.PHPUnit.TestReturnType sniff and are, exclusively
adding the ": void" return types when missing.
This has been generated running the following Sniffs, all
them part of the Moodle's CodeSniffer standard:
- PSR12.Functions.ReturnTypeDeclaration
- PSR12.Functions.NullableTypeDeclaration
- moodle.Methods.MethodDeclarationSpacing
- Squiz.Whitespace.ScopeKeywordSpacing
All them are, exclusively, about correct spacing, so the changes
are, all them, only white space changes.
Only exceptions to the above are 3 changes what were setting the
return type in a new line, and, when that happens, the closing
parenthesis (bracket) has to go to the same line than the colon.
In PHP 8.2 and later, setting a value to an undeclared class property is
deprecated and emits a deprecation notice.
So we need to add missing class properties that still need to be declared.
Default value of the $flag argument changed in PHP 8.1 from ENT_COMPAT to ENT_QUOTES | ENT_SUBSTITUTE
To ensure consistent behavior across different PHP version the second parameter is now required for the functions:
htmlspecialchars(), htmlentities(), htmlspecialchars_decode(), html_entity_decode() and get_html_translation_table()
Applied the following changes to various testcase classes:
- Namespaced with component[\level2-API]
- Moved to level2-API subdirectory when required.
- Fixed incorrect use statements with leading backslash.
- Remove file phpdoc block
- Remove MOODLE_INTERNAL if not needed.
- Changed code to point to global scope when needed.
- Fix some relative paths and comments here and there.
- All them passing individually.
- Complete runs passing too.
Special mention to:
- When belonging to other components and being valid api:
- analytics related tests have been moved to tests/analytics subdir.
- backup & restore related tests have been moved to tests/backup subdir.
- events related tests have been moved to tests/event subdir.
- privacy related tests have been moved to tests/privacy.
- task related tests have been moved to tests/task subdir.
- Some simple renames, not including the component part anymore (not
needed now that they are namespaced):
- some xxxlib_test.php have been renamed lib_test.php
(when they where testing the corresponding lib.php).
- cache stores tests have been all renamed store_test, originally
each one had its own name (file_test, apcu_test, redis_test...)
- assign feedback tests have been all renamed feedback_test, originally
each one had its own name (file_test, editpdf_test...)
Note that, instead of going one by one to them from the XMLDB Editor
(load, makes some tiny change, save, revert the change, save), the
report includes a commented line that, once uncommented, does regenerate
the file while reporting.
I've left it there as a commodity for the developer.
As far as now all them have correct privacy level2 namespace:
- Move them to "privacy" subdir.
- Rename the files to "provider_test.php", this includes old
privacy_test.php and privacy_provider_test.php files
- Rename the testcase to provider_test too (to match file name)
Also, change some relative paths and comments to point to new
locations.
All privacy_test and privacy_provider_test classes:
- Namespaced with component\privacy.
- Fixed incorrect use statements with leading backslash.
- Changed code to point to global scope when needed.
- Renamed a few files to make all be privacy_test or privacy_provider_test.php
- All them passing individually.
- Complete runs passing too.
It's mostly addDocuments(), used by test_add_document_batch_large()
with 100 big documents what requires a lot of ram
although apparently, it's freed (partially) once ended.
Just the peak usage remains high. In any case, isolating that
test to avoid the non-freed side of it to consume too
much memory for the rest. (We allow "only" 2GB).
Still, I think that there is a good work about to detect which
tests are leaking / consuming too much, I'd bet there are
a bunch running completely out of control.
Due to Solr bug SOLR-15039, uploading files for indexing can fail if
it uses multipart upload. This changes it to use direct binary upload.
Unfortunately, the direct binary version in PHP curl only supports a
string, so we have to load the file into memory. I added extra code to
restrict the size of files indexed to (memory limit - 100MB), which is
usually 284MB unless configured differently because cron runs under
MEMORY_EXTRA.
In PHPUnit 9.1, the following regexp-related assertions
have been deprecated and there are new alternatives for
all them:
- assertRegExp() -> assertMatchesRegularExpression()
- assertNotRegExp() -> assertDoesNotMatchRegularExpression()
This is about to, simply, move all cases to the new alternatives.
Source: https://github.com/sebastianbergmann/phpunit/blob/9.1.0/ChangeLog-9.1.md
Regexp to find all them:
ag 'assertRegExp|assertNotRegExp' -li
Both assertContains() and assertNotContains() are deprecated in PHPUnit 8
for operations on strings. Also the optional case parameter is. All uses
must be changed to one of:
- assertStringContainsString()
- assertStringContainsStringIgnoringCase()
- assertStringNotContainsString()
- assertStringNotContainsStringIgnoringCase()
More info: https://github.com/sebastianbergmann/phpunit/issues/3422
Regexp to find all uses:
ag 'assert(Not)?Contains\('
All the setup/teardown/pre/post/conditions template methods
now are required to return void. This was warned with phpunit 7
and now is enforced.
At the same time, fix a few wrong function names,
provider data and param types, return statements...
version = 2021052500 release version
requires= 2021052500 same than version
Why 20210525? (25th May 2021) ?
Because master is going to be Moodle 4.0, to be released
on November 2021. And, until then, we are going to have
a couple of "intermediate" releases:
- Moodle 3.10 to be released 9th November 2020. (2020110900)
This version will be using versions from today to 2020110900
(once it's released the YYYYMMDD part stops advancing).
- Moodle 3.11 to be released 10th May 2021. (2021051000)
This version will be using versions from 3.10 release to 2021051000
(once it's released the YYYYMMDD part stops advancing).
That means that all versions from today to 2021051000 are going
to be used by those 2 "intermediate" releases (3.10 and 3.11).
And we cannot use them in master, because it's forbidden to have
any overlapping of versions between branches (or different upgrade
paths will fail).
So, get that 2021051000, let's add it a couple of weeks to cover
the on-sync period (or a 2 weeks delay max!) and, the first version
that master can "own" in exclusive (without any overlap) is, exactly,
25th May 2021, hence our 20210525.
To support transitions from one search engine to a different one, or
to a different installation of the same kind, this feature allows for
queries to use a different search engine from indexing. So you can
reindex (and do all other search operation) on one server, while
user queries are unaffected on a different server.
This feature supports changing between search engine types, and also
between two Solr installations.
The optimize feature in Solr is usually considered harmful, especially
prior to Solr 7.5.
This change simply removes the optimize implementation from the Solr
engine.
Adding documents in batches instead of one at a time can make
indexing using Solr significantly faster.
This adds new API functions for search engines, including
add_document_batch() to add a batch of documents,
supports_add_document_batch(), get_batch_max_documents() and
get_batch_max_content().
