Commit graph

255 commits

Author SHA1 Message Date
Damyon Wiese
8c2a7f9c38 Merge branch 'wip-MDL-49995-master' of git://github.com/marinaglancy/moodle 2017-12-18 11:35:29 +08:00
Marina Glancy
80364b7b1e MDL-49995 files: delete old file when overwriting 2017-11-28 14:50:20 +08:00
sam marshall
a938e4096c MDL-60174 core_dml: fix miscellaneous incorrect recordset usage
The new recordset support for Postgres requires transactions and
will cause errors if recordsets are not closed correctly. This
commit fixes problems that were identified during unit tests, and
via some basic code analysis, across all core code. Most of these
are incorrect usage of recordset (forgetting to close them).
2017-11-27 11:10:33 +00:00
Eloy Lafuente (stronk7)
796e48a58b MDL-60611 phpunit: remove deprecated stuff and other minor fixes
Thanks @scara for MDL-60611 phpunit: improved redis skipping reasons
2017-11-05 01:45:04 +01:00
Eloy Lafuente (stronk7)
801a372dad MDL-60611 phpunit: switch to namespaced phunit classes 2017-11-02 23:45:34 +01:00
Damyon Wiese
022dccf2a7 Merge branch 'MDL-60001_gzopen' of https://github.com/mrmark/moodle 2017-10-19 13:00:13 +08:00
Eloy Lafuente (stronk7)
4c54ac22d3 Merge branch 'm34_MDL-60318_Undefined_method_file_storage-move_to_trash' of https://github.com/scara/moodle 2017-10-09 23:11:08 +02:00
Matteo Scaramuccia
5ca85a1efe MDL-60318 filestorage: Fixed undefined method move_to_trash() 2017-10-02 22:49:42 +02:00
Matt Clarkson
b1eb6dd7f2 MDL-33174 lib/filestorage: Add filename to curl request
While the lack of the filename in curl POST does not
directly affect Moodle, some external API's expect to get
the filename of the original file, which affects plugins
using the CURL class.
2017-10-02 09:27:51 +13:00
Dan Poltawski
a1b8b9c9b5 Merge branch 'MDL-59960_mimetype' of https://github.com/mrmark/moodle 2017-09-19 09:10:24 +01:00
Mark Nielsen
879827d6fa MDL-60001 core_files: Use local file for gzopen
The gzopen method does not appear to support
file streams.
2017-08-31 15:47:00 -07:00
Mark Nielsen
b757c648f3 MDL-59960 core_files: prevent mimetype update on every file update
Every time a file is updated, the mimetype is re-processed.
But, some mimetypes can only be detected if the file is present
on local disk.  With external file systems, this could cause
the file to be downloaded unnecessarily.  Instead, only update
the mimetype when the file content or the file name has changed.
2017-08-28 11:36:15 -07:00
Mark Nielsen
ee8bce2d3b MDL-59960 core_files: improve mimetype detection
Improve mimetype detection for remote files that
have no file extension.  The mimetype detection
that makes use of the file, only works with local
files, so do not use the remote path which can be
a URL or stream.
2017-08-28 11:36:15 -07:00
Mark Nielsen
9eb1a2c31a MDL-59961 core_files: make content hash validation reusable
Other file systems could make use of this code as they
all need to validate the content has and be able to read
the file from disk.
2017-08-28 11:15:12 -07:00
Cameron Ball
03756f0bdb
MDL-59700 filestorage: Rework repositories to avoid add_file_to_pool
For historical reasons repositories need to call add_file_to_pool
to sync file records. However now that a before_file_created hook
has been added additional information is needed by add_file_to_pool.

Ideally add_file_to_pool and friends will become private/protected,
so we need to remove all uses of it in core.

This patch adds some new methods to the file class to allow syncing
to be managed internally by the file and the file_storage class.
2017-08-07 15:50:15 +08:00
Cameron Ball
8243706542
MDL-59700 filestorage: Add before_file_created hook 2017-08-07 12:27:04 +08:00
gthomas2
023037dc9e MDL-57476 file: Add CUD callbacks 2017-07-14 13:51:51 +01:00
Vladimir Zhirov
ff2dab4566 MDL-59005 resource: fixed zip extraction for filenames in CP-1251/CP-866 2017-06-19 08:50:23 +08:00
Jake Dallimore
e927581a50 MDL-58684 files: Ensure stored_file never serializes the file_storage
Magic methods __sleep and __wakeup used to properly store only the
file_record property and to reinit using get_file_storage().
2017-05-08 11:12:39 +08:00
Andrew Nicols
9020212522 MDL-58718 filestorage: Normalise file paths for Windows 2017-04-26 09:36:10 +08:00
Jake Dallimore
84fb93a77e Merge branch 'MDL-58297-master' of git://github.com/andrewnicols/moodle 2017-04-11 10:49:07 +08:00
Andrew Nicols
a30a04fa01 MDL-58297 filestorage: New functions for hash calculation 2017-04-07 10:40:23 +08:00
Andrew Nicols
941cb7ce3c Merge branch 'MDL-58399-master' of git://github.com/jleyva/moodle 2017-04-05 14:22:39 +08:00
Damyon Wiese
13b449f4a5 MDL-58220 oauth2: cibot cleanups 2017-04-03 13:43:25 +08:00
Damyon Wiese
141ee541ca MDL-58219 repository: Change how controlled links work
Files are copied to the system user as soon as they are uploaded. Write access is then controlled when
serving links to the file.

Part of MDL-58220
2017-04-03 13:41:19 +08:00
Damyon Wiese
151b0f9409 MDL-58132 repositories: Controlled link file type
This introduces a new "controlled link" file type where the file is not
stored in Moodle - but Moodle will control the access permissions on the file.

Plugins can "freeze" a filearea which means Moodle will take ownership of all the remote
files of this type.

When accessing a file, if the "filebrowser" infomation indicates the current user can write to the file, they
will be granted temporary write access.

Part of MDL-58220
2017-04-03 13:40:13 +08:00
Juan Leyva
1104a9fa44 MDL-58399 webservice: Return additional file fields for repositories
We should be detecting when we are treating files that are linked to
external repositories.
For doing that we'd need to return some additional fields via Web
Services:
- isexternalfile
- mimetype (google docs files use an special one)
- repositorytype (the repository plugin name)
2017-03-28 20:23:16 +02:00
Andrew Nicols
1568168717 MDL-55528 fileconverter_unoconv: Add new unoconv plugin
AMOS BEGIN
 MOV [pathtounoconv,admin],[pathtounoconv,fileconverter_unoconv]
 MOV [pathtounoconv_help,admin],[pathtounoconv_help,fileconv_helperter_unoconv_help]
 MOV [unoconvwarning,admin],[unoconvwarning,fileconverter_unoconv]
 MOV [test_unoconv,assignfeedback_editpdf],[test_unoconv,fileconverter_unoconv]
 MOV [test_unoconvempty,assignfeedback_editpdf],[test_unoconvempty,fileconverter_unoconv]
 MOV [test_unoconvdoesnotexist,assignfeedback_editpdf],[test_unoconvdoesnotexist,fileconverter_unoconv]
 MOV [test_unoconvdownload,assignfeedback_editpdf],[test_unoconvdownload,fileconverter_unoconv]
 MOV [test_unoconvisdir,assignfeedback_editpdf],[test_unoconvisdir,fileconverter_unoconv]
 MOV [test_unoconvnotestfile,assignfeedback_editpdf],[test_unoconvnotestfile,fileconverter_unoconv]
 MOV [test_unoconvnotexecutable,assignfeedback_editpdf],[test_unoconvnotexecutable,fileconverter_unoconv]
 MOV [test_unoconvok,assignfeedback_editpdf],[test_unoconvok,fileconverter_unoconv]
 MOV [test_unoconvversionnotsupported,assignfeedback_editpdf],[test_unoconvversionnotsupported,fileconverter_unoconv]
AMOS END
2017-03-10 10:45:46 +08:00
Tony Butler
efa7d24168 MDL-57801 core_filestorage: Delete temp file if disk space runs out 2017-02-16 12:41:06 +00:00
Tony Butler
961323a05f MDL-57801 core_filestorage: Verify hash of temp file before committing
This addresses an edge scenario on NFS filesystems with no space
remaining, where subsequent uploads fail silently while zero byte files
are saved to the pool (and for some reason the filesize check passes).
2017-02-16 12:38:58 +00:00
Andrew Nicols
0c03db6a32 MDL-46375 core_files: Correct filename in mbz test
I noticed during the file system abstraction that this test was
incorrect.

Since both $storagefalse, and $storagetrue are in the same context,
component, area, itemid, and folder, the fact that they had the same
filename meant that they constantly overwrote one another.

As part of archive_to_storage, existing files in the same location are
found, the files themselves deleted, and the existing file record in the
files table is deleted.

The tests continued to pass because:
* the existing variables were not affected by the deletion of the file
  record and file so the comparisons were successful; and
* subsequent calls to fetch the content of the file meant that the
  files themselves were restored from the trash directory.
2017-02-14 08:42:28 +08:00
Andrew Nicols
16a34ae189 MDL-46375 core_files: Split parts of file_storage into new file system
This change moves all operations which deal with the fetching/updating, or
setting of files from the file_storage class into a new file_system class.

A new file_system can be specified in the config.php and used to replace
all relevant methods in order to move the file system component to an
alternative solution.
2017-02-14 08:42:28 +08:00
sam marshall
66234de24c MDL-57515 Resource: 'not very efficient' with a large number of files
Fixes bug where the resource module loads metadata for all files while
building course modinfo, even though it only needs the first file.
(This causes problems if you have ~10k files.)
2017-01-10 17:43:46 +00:00
Eloy Lafuente (stronk7)
a584a403cd Merge branch 'MDL-54916' of https://github.com/mr-russ/moodle 2016-11-07 16:19:06 +01:00
David Monllao
c9af13123d MDL-54751 course_modules: New major upgrade point
Also trailing whitespace cleaned.
2016-11-07 08:59:04 +08:00
Jake Dallimore
3869d77411 MDL-54751 core_course: Edit course libs to support deletion flag
Changed the course libs such that module and section deletion is now
handed off to relevant functions in the util class and exempted
modules scheduled for deletion from the course backups.
2016-11-07 07:41:59 +08:00
Russell Smith
cebce76f61 MDL-56307 course: Refactor file_storage 2016-10-25 10:10:53 +01:00
Juan Leyva
1896b8009a MDL-56307 course: API changes to support retrieve by time 2016-10-25 10:10:52 +01:00
Russell Smith
261bfabc6f MDL-54916 filestorage: Only write empty zip on change.
In some cases we may inspect an empty zip file. If we do open
an empty zip file, check that it changed before we attempt to write
it to disk.  It may be the case that we were reading the file from
a location that we don't have write access to.
2016-10-25 14:10:36 +11:00
Andrew Nicols
b4bb1e65dc Merge branch 'MDL-55445-master-2nd' of git://github.com/FMCorz/moodle 2016-08-17 11:25:24 +08:00
Andrew Nicols
0a0eaa7dfc Merge branch 'MDL-54647-master' of git://github.com/damyon/moodle 2016-08-16 08:01:47 +08:00
Damyon Wiese
300852ee30 MDL-54647 file_storage: Redo the test pdf every time it's viewed 2016-08-15 16:32:21 +08:00
Andrew Nicols
40cc0ba04e Merge branch 'MDL-55513-master' of https://github.com/lucisgit/moodle 2016-08-15 14:07:41 +08:00
Frederic Massart
92b9ef9917
MDL-55445 gdlib: Support for resizing an image 2016-08-15 10:54:28 +08:00
Tony Butler
2ee668ec56 MDL-55513 core_filestorage: Touch and test size instead of file_exists()
file_exists() can return false when file exists (in certain scenarios)
2016-08-10 09:35:23 +01:00
Tony Butler
e98483d748 MDL-55513 core_filestorage: Pass full path of temp dir to remove_dir() 2016-08-10 09:35:23 +01:00
Matt Davidson
826bb3547b MDL-55246 assign: convert files with spaces
If a file is sent to unoconv with a filename that contains spaces,
unoconv will fail to produce the intended results.
2016-08-05 08:14:20 -04:00
Andrew Nicols
2c2f4f17c3 MDL-24343 core: Deprecate unzip_file 2016-08-03 09:05:55 +08:00
Rajesh Taneja
52f3e060e4
MDL-55091 phpunit: Following has been deprecated.
1. getMock()
2. setExpectedException()
3. checkForUnintentionallyCoveredCode renamed to beStrictAboutCoversAnnotation
4. beStrictAboutTestSize renamed to enforceTimeLimit
5. UnitTestCase class is now fully removed.
2016-07-26 10:11:30 +08:00
Simey Lameze
37eafbb6dc MDL-54948 core_files: return false if theres no output from unoconv 2016-06-23 08:37:23 +08:00