S.O.S Blog

You are expected to change variables/details for versions below.   This script will not create the backup directory.  Operator must create those directories manually.

Code is in /var/www/html/

nano up

Paste the following and adjust the variables

# 3.5.1+ (Build: 20180810)
tar -cvf /home/backup/m35/moodle-code-351+-$(date +%Y%m%d%-H%M%S).tar ./;
tar -cvf /home/backup/m35/moodle-data-min-351+-$(date +%Y%m%d%-H%M%S).tar /var/moodle35data/filedir;
mysqldump -u root -p moodle35 > /home/backup/m35/moodle351+-db-$(date +%Y%m%d%-H%M%S).sql;
ls -l /home/backup/m35/;
echo ‘Paused …’;
read $keypress;
git branch -a
php admin/cli/cron.php;
php admin/cli/maintenance.php –enable;
git pull;
php admin/cli/upgrade.php –non-interactive;
php admin/cli/maintenance.php –disable;
php admin/cli/purge_caches.php;
chown apache:apache * -R
fgrep ‘$release’ version.php

To execute:

cd /var/www/html/

source up [ENTER]

Protect up from browsers:

chmod go-rw up

chown root:root up

ls -l up

would look like:

-rwxr—–. 1 root root 803 Aug 15 18:12 up

Now login to the Moodle and go to Site Admin Menu -> Notifications (version is now?) – check updates.   Get updates to plugins.

The first line of this script was aquired via:

fgrep ‘$release’ version.php

while in the code directory.   Added to the top of the script as a reminder.

Posting assumes you can install git via your package manager (yum/apt-get/whatever).  Many distros now included it when distro installed.

Dunno if you have it?   which git [ENTER]

IF the above doesn’t respond with the path to git (normally /usr/bin/git) then your server doesn’t have it installed.

Situation:  have an existing moodle but installed via ftp/wget or some other method of acquiring code.   Code resides in /var/www/html/ and the version of your moodle is 1.9.x

Using Git: “side load”

cd /var/www/

git clone git://git.moodle.org/moodle.git htmlgit

cd htmlgit

git branch –track MOODLE_19_STABLE origin/MOODLE_19_STABLE

git checkout MOODLE_19_STABLE

To see if you are tracking/checked out the 1.9:

git branch -a

The * should be next to the 1.9 branch.

chown apache:apache * -R

We are still in htmlgit.  Now the side load.

cp -rp .git .gitignore .gitattributes ../html/

The above copies (cp) recursively (-r) and preserves (-p) .git (the hidden .git directory and the hidden .gitignore and .gitattributes files up on level (out of htmlgit) and into html … where the moodle code resides.

Now we check the active code directory …

cd ../html/

ls -ld .git (is .git there?)

Now we check which branch it’s tracking.

git branch -a

Should show us:

* MOODLE_19_STABLE
master

with other versions of Moodle under ‘master’.

To make sure git understands issue the following while in /var/www/html/

git reset –hard

The above should result in something like:

Checking out files: 100% (18560/18560), done.
HEAD is now at [lettersnumbers] release 1.9.20+

You are now ready to update the 1.9.x code to the highest 1.9.x version available.

This is a one time thing … using git from this point forward will take place in the code directory of moodle with other git commands.

You can also remove the /var/www/htmlgit directory we used to side load as it is just taking up space now.

 

 

Teachers Plan!
Server Administrators Plan!

Upgrading a 1.9.x to 3.5.x *requires* planning.

Plan your work … work your plan!

If you can, work with a clone of current 1.9.x site and have plenty of
space to backup code, DB, and data directory at every step of the ‘march’.

Along the ‘march’ from 1.9.x to 3.5.x there will be *new* requirements for:
version of PHP
PHP extentions
version of MySQL
Database character set and database collation
Datebase file type

To ‘plan’ the ‘march’, login to your site.
Go to Server -> Admin -> Environment
Update the Component
In the pick list for Moodle version your current version of Moodle will be selected.
What you see below are the ‘Server Checks’.
Green (OK) is good.
Yellow (check) is no so good but not a show stopper, yet.
Red – must perform before attempting to upgrage your current version of Moodle to
the version selected in the drop down.

See:
https://docs.moodle.org/34/en/Environment

How to use the information …
On a sheet of paper, yep, going very low tech here ….

Hand *draw* a chart …

1st Column – Current version of Moodle or the version chosen in the pick list
2nd Column – PHP version currently running
3rd Column – PHP extensions – note only the ones the Environment check says you need
4th Column – Current Version of Data Base
5th Column – Version of Database needed
6th Column – Notes – this for *when* one has to do the PHP version, MySQL version, character set/collation, etc. changes.

Going from 1.9 to 3.5.x you will need *17* rows:
1st Column
1.9.x
2.0
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
3.0
3.1
3.2
3.3
3.4
3.5

Google Sheet 1

Got Google?   Get a copy.

 

Moodle docs:
https://docs.moodle.org/35/en/Cron

First, running cron regularly is a *MUST* … period.

“IMPORTANT: Do not skip setting up the cron process on your server for your Moodle. Your site will not work properly without it.”

Later in the doc, this …

To run the cli cron script every 1 minute, add the line:

*/1 * * * * /usr/bin/php /path/to/moodle/admin/cli/cron.php >/dev/null

NOTE: the final >/dev/null sends all the output to the ‘bin’ and stops you getting an email every 1 minute.

Is there yet a better way? Yes!

touch /var/log/moodle_cron.log

That creates a blank/contains nothing moodle_cron.log file to which one will be sending the cron job output.

Next set up the cron … same time */1 * * * * but we output to a log file.

/usr/bin/php /var/www/html/admin/cli/cron.php >/var/log/moodle_cron.log 2>&1

Breaking that down … the > /var/log/moodle_cron.log redirects to the log file.  The log file is rewritten every run. Thus it never grows where one would have to rotate the log.

The 2>&1. makes the cron job ‘slient’ … meaning it doesn’t send mail to admin user every minute and thus filling up an inbox as well as filling up a drive!

Once that is saved, one can ‘watch’ the cron job in real time via:

tail -f /var/log/moodle_cron.log to see it execute … or error.

If you have multiple Moodles, make multiple cron jobs that log to files for each instance:

Example: a moodle 34 and a 35 on stame server.

touch /var/log/moodle34_cron.log
touch /var/log/moodle35_cron.log

Either tail -f one at a time, or install multitail

https://www.vanheusden.com/multitail/

Once multitail installed, one couuld watch both crons at the same time:

nano /usr/local/bin/watchcrons

In ‘watchcrons’, a single line:

/usr/bin/multitail -i /var/log/moodle34_cron.log -i /var/log/moodle35_cron.log

Save the watchcrons file. Make it executable.
To execute: watchcronS [ENTER]

Also note: multiple moodles means multiple crons … rather than setting them both up to run every minute, one might have to give a little wiggle room between running.

Every 2 minutes – 1 minute apart … why? Some of the jobs use up M’s of memory. How much memory do you have? Matter of fact, the docs for Cron at Moodle.org even suggest that for larger sites, a dedicated server for the running of crons. Posssible to do since version 3.4 of Moodle.

Query shared by Rick Jerz on Moodle forums.

Turned it into a bash shell script so when working from command line one can see how many users are logged onto moodle during the last 5 minutes.

Called ‘last5′ … here’s the contents (all on one line – you sub the [USER] [PASSWD] and [DBNAME] below:

mysql -u [USER] -p'[PASSWD]’ -e “use [DBNAME];SELECT firstname,lastname,lastip,from_unixtime(lastaccess) as LastClick,now()-interval 5 minute as Last5Minutes FROM mdl_user having LastClick > Last5Minutes order by lastaccess desc;”> last5.txt;cat last5.txt

Outputs something like:

firstname lastname lastip LastClick Last5Minutes
Ken Task x.x.x.x 2018-08-23 19:20:00 2018-08-23 19:16:10

 

bashmdlscripts

above linked to bashmdlscripts.zip

Contained in the zip:

checkvars – checks if you have filled in variables in mdlvars.txt
dobu – backs up one course
getcids – gets course ids to use with dobu
getcourses – gets a courses.txt file that list courses by id/shortname/longname
mdlvars.txt – set variables for these scripts
niclbackups – non interactive command line backups
sbackupcode – all the s- scripts use mdlvars.txt file for variables. backup for code
sbackupdata – backup data
sbackupdb – backup database

To use these, first upload the zip to /path/to/your/moodle/code/admin/cli/
Unzip the zip there.

Strongly suggest you open with text editor and inspect each or at least ‘cat’ them to view contents.
Example: cat dobu

* Edit the mdlvars.txt file providing the variables first

** Backup directories must be created manually … these scripts will not create them
for you.

Offered as is … no support will be provided.

‘in the spirit of sharing’ …

For Moodle Servers, don’t think GDPR applies to K12 entities nor Colleges and Universities, but it sure does with Training Depts in Corps.

Moodle has always lacked robust reporting – core or addon/plugin. Besides, that if there is a commercial addon/plugin, do you really need it? Maybe.
But for things like GDPR ‘discovery’, there is a very affordable way to determine if one should be concerned … via command line … a poor man’s report .. if reader wills.

In moodlecode/admin/cli/ create a exeutable bash shell script that is really
a query of the Moodle DB that outputs to a text file.

Name: euusers

Contents: (you change DB user and Pass as well as DB name and it’s all on one line.

mysql -u [DBUSER] -p'[DBPASS]’ -e “use [DBNAME];select id,firstname,lastname,email,country from mdl_user WHERE country RLIKE ‘GB|FR|IT|BE|BG|CZ|DK|DE|EE|IE|EL|ES|HR|CY|LV|LT|LU|MT|NL|AT|PL|PT|RO|HU|UK’;” > euusers.txt; cat euusers.txt|more; wc -l euusers.txt;ls -l *.txt

To execute the script as root, either use:

source euusers

or if you could make the ‘euusers’ script executable:

chmod u+x euusers

Then you could use ./euusers to execute.

What the script does …
* searches mdl_user table for users who have an EU nation in their profile.
* exports those to a euusers.txt file
* shows you the euusers.txt files and does a ‘word count’ on that file so that
you have the number of users.

Output looks like:

id firstname lastname email country
3104 bob jones bob.jones@somenet.com HU
3235 mary jones mjones@somenet.com FR

to make this available to System Admins of the Moodle, one could
direct the export of the file to a file system repo in moodledata/repository/serverinfo
And in courses reserved for System Admins, setup the use of that file system repo so that those courses can link to the .txt files contained therein.

So we have moodledata/repository/serverinfo/
We change the redirect line in the script above to:
> /full/path/to/moodledata/respository/serverinfo/

Now one can link to that text file report in the Moodle interface.

For this, one needs a Google account and to have setup a Storage Bucket
https://cloud.google.com/storage/docs/creating-buckets

Info below (in a nutshell) came from this resource:

https://www.assistanz.com/mount-google-cloud-storage-bucket-linux/

Well done and easy to follow.

now have gbucket on a Rackspace server and use the same Google bucket on
a Google Compute Engine instance as well.

Didn’t set in fstab as that might stall a reboot of server IF bucket can not be mounted.
Instead have a ‘mountgbucket’ script in /root that mounts.

Here’s the quickie …

wget https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-156.0.0-linux-x86_64.tar.gz
tar zxvf google-cloud-sdk-156.0.0-linux-x86_64.tar.gz
./google-cloud-sdk/install.sh
source /root/.bashrc
gcloud auth login
gcloud components update

———————————————
* if you have already attempted setting up a bucket, one might
have to remove the following after doing the gcloud components update command.

rm /usr/lib64/google-cloud-sdk/bin/gcloud
rm /usr/lib64/google-cloud-sdk/bin/git-credential-gcloud.sh
rm /usr/lib64/google-cloud-sdk/bin/bq
rm /usr/lib64/google-cloud-sdk/bin/docker-credential-gcloud
rm /usr/lib64/google-cloud-sdk/bin/gsutil
gcloud components update
——————————————–

gcloud auth list
gcloud auth application-default login
gcloud auth application-default print-access-token
cd /etc/yum.repos.d
nano gcsfuse.repo

Contents of gcsfuse.repo
[gcsfuse]
name=gcsfuse (packages.cloud.google.com)
baseurl=https://packages.cloud.google.com/yum/repos/gcsfuse-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0

yum install gcsfuse
mkdir /mnt/gbucket
gcsfuse [yourbucketname] /mnt/gbucket
fusermount -u /mnt/gbucket

cd /root/
nano mountbucket

Csontents of mountbucket

export GOOGLE_APPLICATION_CREDENTIALS=/root/.config/gcloud/application_default_credentials.json;
mount -t gcsfuse -o rw,allow_other,nonempty [yourbucketname] /mnt/gbucket

Save the file
Make it executable:
chmod u+x mountbucket

Now the big test … mount it

./mountbucket/
It will display info and success.

See it there:

df -h

One should see a line:

yourbucketname 1.0P 0 1.0P 0% /mnt/gbucket

cd /path/to/moodledata
and in moodledata issue:

du -cksh *

Example output:

25M cache
96K environment
53G filedir
24K lang
9.3M localcache
920K lock
228M mdeploy
20K muc
320K sessions
144K temp
4.0K trashdir
56G total

Have removed from example above directories that are specific to the
server used to share this info, so the total at the bottom isn’t accurate.

Handy to know if one is thinking about running a full clone on same server.