Installing and updating Drupal 9 on a shared web hosting platform for Mac user (Part 3)

From the last part of this tutorial, we setup a new repository and push the full Drupal files and directories into the new master repository. 

Now we want to grab this repository and setup the same project on our computer. This is what is called cloning a repository because we want to the exact same files, directories and structure to work with. 

Let's test to see if this is working. We will clone the repository to our local machine. First let's navigate to our MAMP root directory. Open a terminal and type:

cd /Applications/MAMP/htdocs

We can now begin to clone it into a directory of your choice. For the sake of this tutorial i will name the new directory 'test2'. You can name it whatever you want to your own setup. If you have been following my last tutorial, you would have created a directory for your initial site setup where we push all the changes to the master repository. Do not delete this directory as it will come in handy later. 

Let's clone and create this new directory where the site can reside in. Type:

git clone https://github.com/[username]/drupal.git test2

You can get the url if you visit your Github repository for this project. You need to click on the green button that says 'code'. 

Note: Replace [username] with your own. 'test2' is what i named the directory, it is up to you whatever name you want for your directory.

Once completed, you will notice files and settings.php is missing. That's because we deliberately ignore these files and directory since we can clone this to any other server we want. You local and live site will probably have different login credentials for database and files maybe different over time. In addition, you maybe have different config settings stored in settings.php. So it is not recommended to version control the same file that is unique to a specific domain. 

Again, remember the initial site we setup from previous tutorial. Since we have not modified any of the files which means this is the exact same installation copy. We can copy over the files directory and settings.php file to this new repo (which i named it 'test2') in the same directory structure. It will use the database you have initially setup since the login credentials since this is already in the settings.php file you copied over. But before you can view it in your web browser, you need to create a new vhost. We have gone over creating a new vhost in the last tutorial.

Adding Drupal modules

Now that we know the clone of the repository as our local branch is all working fine. We can start modifying this local branch by adding a couple of modules. I always add admin toolbar module.

To add a module using Composer, run this command in your Drupal root installation directory:

composer require 'drupal/admin_toolbar'

This will get the latest stable version. 

Login to your Drupal site and enable the module. If it is working fine, we can commit the admin toolbar module files and push our changes to master remote repository. 

Let's see what files has been added or modified, run this command:

git status

This will print out the current status of your local branch so you should see something like this:

On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   composer.json
	modified:   composer.lock
	modified:   vendor/composer/autoload_files.php
	modified:   vendor/composer/autoload_static.php
	modified:   vendor/composer/include_paths.php
	modified:   vendor/composer/installed.json

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	web/modules/contrib/

no changes added to commit (use "git add" and/or "git commit -a")

As you can see, Git is telling us that we have about 6 modified files. We need to add all of these files. Run each line separately:

git add composer.json
git add composer.lock
git add web/modules/contrib/

git status

 The line git add web/modules/contrib/ will add any modified files Git finds inside contrib directory recursively and add them all ready to be committed into local repository.

For some reason, now we also have the following to add

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   vendor/composer/autoload_files.php
	modified:   vendor/composer/autoload_static.php
	modified:   vendor/composer/include_paths.php
	modified:   vendor/composer/installed.json

type:

git add vendor/

We can commit our changes. Type:

git commit -m "added Admin Toolbar module"

To push commits made on your local branch to a remote repository, type:

git push -u origin master

That's it done. These newly modified files are in the remote repository so next time when you clone the repository, you should also have your newly modified files as well. You can also visit your repository on Github.com and view the changes made to your repository. 

Setting up and updating live site with Git

Before we setup live site, we need to setup the database to use. you can use phpmyadmin in MAMP to export local database. However, if this failed then you can do it via command line. This tutorial: Importing/Exporting a Large Database in MAMP you help you.

Next, if you have phpmyadmin or any other tool to manage you database running on your hosting server, you can use that to import your database. Otherwise, you need to do it on the command line. Follow this guide on how to import your DB

Next, open up a terminal and SSH to your hosting server:

ssh -p 8822 [email protected]

The -p option is port. If your hosting provider uses the default port (ie, 22) then you don't need to add -p 8822

username is your username to login to your server. domain.com is the domain of your website. Replace them with your own.

Once your database is setup, we can begin to clone your repository to create a live site.

In your terminal (you should already be logged into your server if you have followed the above), make sure you are in a directory above the /public_html directory (usually the /home directory) and type:

git clone https://github.com/[username]/[your_repo] public_html/

replace [username] with your own and [your_repo] should be the name of your repository you want to clone from Github. The /public_html directory is the location where the repository will be clone into. In most case public_html is the web root folder of your web server. 

Once git has completed cloning, you need to copy your web/sites/default/files folder and settings.php from local installation to live site in /public_html/web/sites/default/ 

Copy /vendor to your webroot (ie, public_html/).

Now open up /public_html/web/sites/default/settings.php and update the database login details to match the one you have on your live site.

$databases['default']['default'] = array (
  'database' => 'db-name',
  'username' => 'db-user',
  'password' => 'db-password',
  'prefix' => '',
  'host' => 'localhost',
  'port' => '3306',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
  'driver' => 'mysql',

Open up a web browser and view your live site. It probably will fail and give you this error notice:

Redirects to external URLs are not allowed by default, use \Drupal\Core\Routing\TrustedRedirectResponse for it.

This is the gotcha moment where most people stumbled on . As you remember, we said that the drupal webroot is /web so the site needs to be serve from this directory, not /public_html as this should now be treated as outside of the drupal installation web root directory. If you were running a VPs and have root access (admin privileges), we would create a vhost in the Apache web server configuration file to make drupal web root as the root directory for your domain. Unfortunately, shared hosting server platform does not allow this. So we need a workaround and that is to create an .htaccess file to redirect to /web as the main web root. We need to create this in your /public_html directory

In your terminal type:

cd public_html
touch .htaccess && nano .htaccess

This will create a new file called .htaccess and open it up in Nano (command line editor).

Copy this code into your .htaccess file:

<FilesMatch "\.(engine|inc|install|make|module|profile|po|sh|.*sql|theme|twig|tpl(\.php)?|xtmpl|yml)(~|\.sw[op]|\.bak|\.orig|\.save)?$|^(\.(?!well-known).*|Entries.*|Repository|Root|Tag|Template|composer\.(json|lock)|web\.config)$|^#.*#$|\.php(~|\.sw[op]|\.bak|\.orig|\.save)$">
  <IfModule mod_authz_core.c>
    Require all denied
  </IfModule>
  <IfModule !mod_authz_core.c>
    Order allow,deny
  </IfModule>
</FilesMatch>

RewriteEngine on
# Change "example.com" to your own domain name:
RewriteCond %{HTTP_HOST} ^(www.)?example.com$
# Change "directory" to the directory where Site is installed:
RewriteCond %{REQUEST_URI} !^/web/

# Change "directory" to the directory where Site is installed:
RewriteRule ^(.*)$ /web/$1
# Change "example.com" to your own domain name:
RewriteCond %{HTTP_HOST} ^(www.)?example.com$
# Change "directory" to the directory where Site is installed:
RewriteRule ^(/)?$ web/index.php [L]

Change 'example.com' to your own domain name.

Note: For this line: RewriteRule ^(/)?$ web/index.php [L]

I use [L] and it works fine for me but you might want to consider using [R] if you are experiencing some issues. Explanation is below.

"[R]" is to redirect the path and will show "web/" as part of the redirected path. When using "[R]", "RewriteBase /" is required to add "/" before "web/$1". Without this, "/var/www/html" will be added before "web/$1" and will cause unexpected results. This effect was discovered after many days of error discovery using "[L]".

Using [L] as suggested by many people may hide "web/" from the displayed path, and does not require the use of "RewriteBase", but some of the web pages will still show "web/" unavoidably. A mixed use with or without "web/" displayed will cause denial of access rights or redirection to external URL in some cases. Therefore, it is better to force to display "web/" using "[R]".

[Source: https://kctang.com.hk/web/install-drupal-9-content-management-system]

 

We have one more changes to make. Open up your /public_html/web/sites/default/settings.php file and add this code to the bottom of settings.php file: 

/* Fix redirect issue to subdir (/web) on drupal 8/9 */
if (isset($GLOBALS['request']) and
'/web/index.php' === $GLOBALS['request']->server->get('SCRIPT_NAME')) {
$GLOBALS['request']->server->set('SCRIPT_NAME', '/index.php');
}

That should fixed the redirect issues. Visit your website and refresh the page. It should now work as expected.

We want to commit this file, type:

git add .htaccess
git commit -m "add htaccess to redirect to /web as web root"
git push -u origin master

 

Now that we have a clone of the site from the git repository, we can make changes like we did on our local repository on the local machine. But this is a live site so it's recommend to leave it as is and make any changes on the local repository to minimise any disruption while site visitors are viewing your site. 

To test if you can pull in the new changes from repository, we can go ahead and add Backup migrate module. We will need this if you plan on backing up your files directory and database which i recommend since git does not version control your database. 

To do this, go back to your local installation on your terminal change directory to where your composer.json file is and type:

composer require 'drupal/backup_migrate'

Enable and test the module. If it is working fine, we can commit and push our changes to the remote repository. 

git add composer.*
git add web/modules/contrib/backup_migrate/
git add vendor/composer/installed.json

Once added to repository we can commit our changes and push to remote repository 

git commit -m "added backup migrate module"

Finally, push our changes:

git push origin master

SSH to your live website server in your terminal. We can pull in the new changes made on local machine since we have modified, commit and push the changes to remote master repository.

In your /public_html directory, type: 

git pull origin master

Go back to your web browser to view your site and run: /update.php and clear the cache in your Drupal site.

Removing a Drupal module

To remove a module, make sure the module is uninstalled from Drupal admin UI first. Then use git (for example, we want to remove Admin Toolbar module):

git rm -R web/modules/contrib/admin_toolbar

Remove it with Git before Composer. So next step would be to go into your composer.json file and delete the entry in required for the module that has been removed. In this example, it was the admin_toolbar.

You ca run:

composer update --lock

To update the composer.lock file.

 

I hope this tutorial is enough to get you started on managing your site using Git on a shared hosting server.

Before i wrap up this tutorial, some important things to take note of:

  • Bear in mind, as your site grows both web traffic and resources, it could easily outgrow your shared hosting server to handle the site. So your site may start to experience slow response time. If you can go with a VPS server with root access, i would highly recommend this option because it also means you can install better caching system on the server for better site performance. Basically, You have more flexibility to optimise your server. 
  • When you add or delete files and directories, do it with Git and not manually. This is because Git is keeping track and version controlling your project. so it wouldn't be able to keep a record and compare of what is in the repository. 
  • If you have many files to add, you can use: git add --all 
    This will add all files that have been modified.
  • When you remove a module, you need to uninstall it first from Drupal admin UI then remove it from your installation directory. 
  • To remove files, use: git remove [filename]
    Replace [filename] with the file name you want to remove.
  • Don't forget to update your composer.json to remove the module there as well.
  • Sometimes your composer.lock file is not as up to date as your composer.json. To ensure it remains up to date, run composer update --lock 
  • Composer cheatsheet
  • Don't develop and test on live site. Always, do your development on a local repository and only pull in the new changes on live site. Once you deploy your changes to live site, remember to run /update.php and clear the cache. 
  • If you messed up badly, you can delete the complete local repository and clone the repository again to work on but be very careful any new changes will be lost (unless you have committed the change to your remote repository).
  • It is recommended to install Drush:
    composer require 'drush/drush'
     To run Drush on command line, navigate to your web root (the directory above /web) and type (as an example): 
    vendor/drush/drush/drush updatedb
    This will run /update.php to update the database.
    To clear the Drupal cache, type:
    vendor/drush/drush/drush cr

     

  • Database are not version control by Git so be sure to setup a daily backup. This can be done using Backup migrate module.
  • If you need to deploy changes made to your Drupal installation and stored in database, this is what Drupal configuration manager is for. Remember, always store your config files outside of the web root (you will find this in your settings.php file): 
    $settings['config_sync_directory'] = '../config/sync';

     

  • Learn how to create a branch to work on part of a development then merge your changes once it is ready. A branch can be a particular feature or bugfix you are working on.
  • Some times you will find that the same site with all the same modules fits for a new site build though content will differ greatly. Then there is nothing to stop you from deploying the same build in your remote repository to a different domain to reuse and create a new site out of it.
  • The real strength in Git is collaboration with other developers. You will see how useful it will become once your team grows and everyone is working on a set of feature or build from the same code base. 
  • A really good resource to learn Git in more depth is https://docs.gitlab.com/ee/gitlab-basics/start-using-git.html
Eliezer (not verified), 13 Dec 2020 - 5:03pm
This seems to be very dangerous, since your .htaccess doesn't include any rules preventing people from accessing the files in /public_html/ directly.

I ended up using this .htaccess instead, which forces ALL requests to get redirected to /web/:

# Change "example.org.il" to your own domain name:
RewriteCond %{HTTP_HOST} ^(www.)?example.org.il$
RewriteCond %{REQUEST_URI} !^/web/
RewriteRule ^(.*)$ /web/$1

# Change "example.org.il" to the directory where URL of the website:
RewriteCond %{HTTP_HOST} ^(www.)?example.org.il$
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^(/)?$ web/index.php [L]
Duvien, 14 Dec 2020 - 9:51am
I don't see a difference except the lines are placed in different ordering and this extra line added: RewriteCond %{REQUEST_URI} !=/favicon.ico which is just the favicon image.
Eliezer (not verified), 14 Dec 2020 - 10:19am
We removed these lines:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d

Which were allowing users to access the file **/public_html/composer.json** file, and list the contents of directories, and all sorts of fun things.

We iterated it over a bit more, and this is our real, final **/public_html/.htaccess**:

RewriteEngine on
# Change "example.org.il" to your own domain name:
RewriteCond %{HTTP_HOST} ^(www.)?example.org.il$ [NC]
RewriteCond %{REQUEST_URI} !^/web/
RewriteRule ^(.*)$ /web/$1

....And that's it. The .htaccess inside **/public_html/web/.htaccess** handles the rest, including redirecting to index.php, and managing security to prevent unauthorized file access.
Duvien, 14 Dec 2020 - 11:11am
Thanks, that works. I should have tested my own htaccess for security so thanks for raising this concern to me. I also added to the top of the htaccess:

<FilesMatch "\.(engine|inc|install|make|module|profile|po|sh|.*sql|theme|twig|tpl(\.php)?|xtmpl|yml)(~|\.sw[op]|\.bak|\.orig|\.save)?$|^(\.(?!well-known).*|Entries.*|Repository|Root|Tag|Template|composer\.(json|lock)|web\.config)$|^#.*#$|\.php(~|\.sw[op]|\.bak|\.orig|\.save)$">
<IfModule mod_authz_core.c>
Require all denied
</IfModule>
<IfModule !mod_authz_core.c>
Order allow,deny
</IfModule>
</FilesMatch>

Will update my article to reflect this.
Tac_bb (not verified), 28 Apr 2021 - 12:28pm
Thank you for sharing this. It has been a great help.

Just incase someone finds this useful, I also added a bit to the .htaccess in public_html to enforce https and redirect www to non-www. (Putting this within the .htaccess in public_html/web caused problems.)

RewriteCond %{HTTPS} off
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
RewriteRule ^ https://%1%{REQUEST_URI} [L,R=301]
Dave (not verified), 18 Dec 2021 - 8:12pm
Genius! I recently had to get a Drupal 9 site loading on shared hosting in a public_html folder. You saved me a lot of pain! Thank you.
The content of this field is kept private and will not be shown publicly.
Your email address will be kept private and will not be shown publicly.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.