Due to Solr bug SOLR-15039, uploading files for indexing can fail if
it uses multipart upload. This changes it to use direct binary upload.
Unfortunately, the direct binary version in PHP curl only supports a
string, so we have to load the file into memory. I added extra code to
restrict the size of files indexed to (memory limit - 100MB), which is
usually 284MB unless configured differently because cron runs under
MEMORY_EXTRA.
To support transitions from one search engine to a different one, or
to a different installation of the same kind, this feature allows for
queries to use a different search engine from indexing. So you can
reindex (and do all other search operation) on one server, while
user queries are unaffected on a different server.
This feature supports changing between search engine types, and also
between two Solr installations.
The optimize feature in Solr is usually considered harmful, especially
prior to Solr 7.5.
This change simply removes the optimize implementation from the Solr
engine.
Adding documents in batches instead of one at a time can make
indexing using Solr significantly faster.
This adds new API functions for search engines, including
add_document_batch() to add a batch of documents,
supports_add_document_batch(), get_batch_max_documents() and
get_batch_max_content().
Adds new API support within search engines for optional methods to
delete data for courses and contexts, and implements this for the
two core search plugins (simpledb and solr).
The new API is automatically called when courses or contexts are
deleted. When a whole course is deleted, it only sends the course
delete rather than sending 1,000 separate context deletions as
each activity/block is deleted.
Implements a mechanism by which search engines can provide different
result orderings, and implements a 'by location' ordering within the
Solr search engine (available whenever the user starts their search
from within a course or activity).
Adds group support to the core search API and the Solr search engine.
This allows for:
* User searching by group (in the API only, no interface yet)
* Automatically restrict search results by group (in some cases like
separate-groups forums)
- Silents the system call when no correct data provided
- Converts long exception messages to standard server
error + debugging message with the exception
Also removing is_server_ready call from execute_query as in production
workflows search is done through \core_search\manager::search and
\core_search\manager::instance already checks is_server_ready. In
testing environment setUp functions should check that the server is
ready before performing other actions.
Ensures that Solr will return available results, even if there are many
misses when using check_access(), by asking Solr for more results if the
counter says there should be more.
Improves performance by ending processing as soon as the requested page
of results is processed. Remaining number of pages is an "estimate"
based on the total result count from Solr and how many items we have
rejected up to this point.
Expand the Solr engine to use the built in (to Solr) Tika
file indexing engine. Files are shipped off for indexing with
curl. Each file must have it's own document. So grouping is
used to keep files and Moodle docs associated.
