Composer based Drupal projects can be fun

Submitted by Jürgen Haas on Mon, 01/09/2017 - 17:25

As a team that had spend time and money for years in developing best practices for staging, and operations of many sites on version 5 to 7, the all new Drupal 8 came with some surprises and challenges in 2015, when first alpha and then beta releases started to see the light. Where all of Drupal core  up to Drupal 7 had been exposed in the main project folder, all the contributed code from the Drupal community and third parties was located deep down in the code tree with custom code for the specific project sometimes was hard to separate from other contributions. Consistency was just a dream, e.g. contributed installation profiles had to be in the same folder as installation profiles from core.

service providers got used to that. We even managed the odd scenario where someone had placed an add-on in a profile's subdirectory, in core's module directory or even in a modules directory of one of the sites hosted by that Drupal installation. And of course, the aforementioned best practices were sophisticated enough to take care of all those Drupal flavored oddities.

Then this Symfony based 8 beast came around the corner and let alone the directory and file structure had completely changed - much to the better, just to make that crystal clear right upfront. Here are the highlights:

  • core went into a core subdirectory and as long as you're not hunting bugs or even maintaining core, you no longer have any reason to ever again look into that folder. This is core, it is great, it does its job, it's there for you. And you can leave it as is. All your , additions and what more goes into different places. Nothing to do with Drupal core anymore. Fantastic.
  • A new directory called vendor arrived with a huge number of third party packages or libraries from the PHP community that not only serves but also all sorts of other PHP based platforms and applications. Again, this is a place where a lot of great code is living, but nothing that most Drupal agency have to worry about while building web applications based on Drupal.
  • Right at the center of a project, there are 4 directories which are for us. Here is where the additions are installed and is happening. Right in the root of the web project, not deep down in the tree:
    • /libraries
    • /modules
    • /profiles
    • /themes

Just looking at that structure at a first glance, this was so exciting to me. It's been so obvious and I couldn't agree more: this was a great step in the right direction. NB: Yes, I know that 8's re-architecture has been much more than this and I'm far from trying to down-play that achievement, it's just that this blog post focuses on the resulting structure and what that meant for our workflows.

But wait, how do I update core then?

What looked so great right at the beginning, caused major headaches a couple of weeks later, when the next alpha release was made available and suddenly highlighted the fact that the previously successful update and strategies wouldn't work anymore. Calling pm-updatecode didn't work and threw error messages all over the place. Running wasn't something I had done too often before and I didn't even know how to first update the .json file to get started. At least that approach didn't seem to make sense. Download the new core tar file and copying that over the existing code tree? Somehow a terrifying approach that I never felt was anywhere near to my keyboard and terminal.

Reaching out to the community at that point was a challenge. Everyone has been super busy. Digging deeper it became more and more obvious that this question had not been addressed yet: how does the life-cycle of a 8 site look like.

The best advise at the time was this: download the latest release of core, extract that in a temporary directory. Then delete the /core and /vendor directories in your web root and move those 2 directories from the just extracted archive into your project. After that run a series of different commands and with a bit of luck, you should be ready to go with an updated core. What?

Right, there is a bit of research, trial and error, discussions and some projects with real life experience required to get to an all new for maintaining sites from a staging and perspective. And many Drupal specialists have worked on this for the last 12+ months with the conclusion that a based workflow is not just the best we can get, it really makes a lot of sense - and it's great fun!

is a great tool

To my surprise there is currently a lot of negative stories out there in the wild about , especially in the community. This is mainly driven by the fact that we as a Drupal community currently have to learn quite a lot of new things, all at once. We've all heard about or even got involved in this huge effort of Drupal 8 that took several years. And some great people even left the community because they disagreed with the approach and felt that the object-oriented architecture would be too big of a hurdle for the many self-educated developers.

From my own experience I can only tell, the adventure is amazing. You only need to let go and be prepared to learn new stuff. Once in there, developing services, plugins and controllers is so simple and you can get great support from IDEs and other tools.

In this context, all of the sudden we also get hit by . It needs our attentions and steels our productivity. I believe that this is the reason why we currently read so much frustration about it in our community. This will change to the better. Quite soon, as far as I'm concerned.

Reading a lot on how others are doing things, I came across a nice project on GitHub that started a Composer template for Drupal projects. It worked well straight away and what became obvious very quickly was the fact that the directory structure of a project would get changed again - once more to the better.

Taking that template as a blue print and trying to fit that into our own which is influenced by some and Ansible components, we started to tailor that towards a solution that meanwhile got into production in half a dozen projects and works like a charm. You can find the project at Drupal 8 Project Template on our repository.

Here is how such a structure looks like:

- Project Root
  - /config
  - /console
  - /
  - /files
  - /settings
  - /vendor
  - /web
    - /core
    - /libraries
    - /modules
      - /contrib
      - /custom
    - /profiles
      - /contrib
      - /custom
    - /sites
    - /themes
      - /contrib
      - /custom
    - index.php
  - .gitignore
  - .json
  - composer.lock

There are only a couple of changes from the structure of 8 described above:

  • There is a project root with the root being a subdirectory of that which is named /web
  • The /vendor directory is now in the project root and no longer in the root
  • The 3 directories /config, /files and /settings are on the project level as well and that way clearly separated (more to that in a minute)
  • The tools ( and ) have their own locations too

All the contributed modules, themes, profiles and libraries got into their /contrib directories in the root and our custom code for the project also has their own subdirectories in there.

What goes into

From the structure above, what's specific to a customer project is pretty clear:

  • .json and composer.lock
  • config and settings
  • all of the custom directories

Everything else is reproducable and in no way unique to any specific client project. For that reason, we've decided to only commit those components into the customer project repository. But we do not want to conceal that there are discussions around that topic going on in the community. Just mentioning that so that you are aware that the described approach is we've selected, it's certainly not the only possible one.

Reproducable? Yes, from just the 3 components listed above the whole site can be setup, ready to go either for production or for another developer joining the team. Almost. What's missing in the repository and still isn't reproducable is the files directory. This is where images and documents are stored and those are managed differently in our . As there is a lot of binary data most of the time, and pretty big files as well, there are various options on how to handle those file. This is not within the scope of this blog post and we'll cover that elsewhere. BTW, the same applies to the database.

How to use all this: the

All of what's in the repository is what you need to setup the site. Very lightweight and nice to handle. But before describing how to setup the site on a different host, let's have a look how to get started. It requires only one single command:

 create-project lakedrops/d8-project [DIRNAME] --stability dev --no-interaction 

Just replace [DIRNAME] with the destination, where to create the new project and a few minutes later, you've got a fully functional environment and of the customer project can begin right away. Even the repository got created locally and you can push that to any remote repository of your choice. Source code of the project templates is and you can find it, clone it or contribute to it over at our GitLab repository.

When ever you want to setup that project on another host, either for another developer, for or even for production, simply checkout the project from your repository and call the following command:

# Production environment
 install --no-dev --optimize-autoloader --prefer-dist

#  environment
composer install --optimize-autoloader --prefer-dist

This is installing all components ( core, modules, themes and all vendor packages) in one go and uses exactly the same version for each of those components that you've committed to in you environment. This is possible because stores all details about installed components in its .lock file so that installing from that repository will give you exactly the same result as last time.

To update existing components or to add new ones, we edit compose.json in our environment and then call this:

 update 

That updates all the components and their dependencies automatically and updates the details in .lock again. Committing to and re-deploying on your other hosts like described above is all you need to do. OK, to be fair there are additional tasks like running 's update routines, flushing cache and other maintenance work and we strongly advise to also automate all those tasks. But those are required in any case, not only within a based environment and therefore won't be covered by this blog post.

This is pretty much everything that's required. The full for a agency also needs to define the flow for settings, configurations, files, databases, stages and other areas. Those are interesting topics too, but the part of the workflow is based on the building blocks described above. We think, this is enjoyable, hope you have fun with it too.

 

 

   

Comments

Kai Gertz (not verified)

Thu, 01/26/2017 - 15:39

Hi Jürgen - it's been a while since you first presented your composer approach in our local usergroup. As we are currently starting a new project with Drupal 8 I decided to check it out again for that purpose. Now I just started to understand the magic behind the scenes (those composer plugins that do the scaffolding and theme setup) - really useful stuff, thanks for sharing!

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.