...
| Note |
|---|
This MUST be in place before the Mautic 4 Beta Sprint (23-25 April) |
Research
Splitting the repos while maintaining the GitHub commit history (this does not take into account maintaining the sync)
There is a built in, GitHub recommended way to do this: https://docs.github.com/en/github/getting-started-with-github/splitting-a-subfolder-out-into-a-new-repository - we should could follow this process for the plugins and ensure that we end up with the correct branching strategy
Try this with one of the plugins and see how this works - then maybe roll that out more widely
Rahul Shinde is working on a POC for this in https://mautic.atlassian.net/browse/TPROD-204
Maintaining the sync between mautic/mautic and the core-lib, plugins and theme repos
Once we have split out the app directory, plugins and themes we then need to figure out how to keep the app directory, plugin and theme folders in sync
If we physically remove the files from the main repo, we then need to figure out how to manage PR’s for those repos:
Do we accept them to core?
How, if the folders do not exist in http://github.com/mautic/mautic ? I think this might not be possible if we remove the folders entirely.
Do we accept PR’s to plugin repos?
This would be really clunky if a bug fix needs to apply to several different plugins or to core and to the plugin - you would have to make multiple PR’s, know how to find the right repo, etc. Pretty poor/confusing developer experience?
There are a few tools out there that we could potentially consider:
https://github.com/dflydev/git-subsplit [Recommend to not consider due to no recent updates]
Not sure if this is what we are after - it mentions a subtree split. Need to read more.
Allows you to do the subsplit and maintain thereafter
One-way read only sync
Allows you to specify the heads/tags to be updated if you don’t want to auto-detect
Seems to be abandonware - 4+ years since last update and multiple open issues/PR’s
https://github.com/ryanwinchester/subsplit-service [Recommend to not consider due to no recent updates]
Allows you to do the subsplit and maintain thereafter
Composer-based install
Also supports webhooks if we need to use them
Seems to be abandonware - 5+ years since the last update
https://github.com/jderusse/docker-gitsplit was mentioned by Tobias Nyholm (maintainer of subtreesplit.com/symfony core team)
Allows multiple branches
Allows you to use Github Actions
Splits are all maintained in a workflow file
Has to clone the entire repo - may have performance issues?
Triggered on push to main branch and on release, based on example - so should meet our needs
https://github.com/thephpleague/flysystem/blob/2.x/.github/workflows/publish-subsplits.yml was mentioned by Tobias Nyholm (maintainer of subtreesplit.com/symfony core team)
Github Action used by Flysystem to push subsplits automatically - could be something we could look into but may need some assistance from Dennis Ameling to understand how we might use it on PR merge / release and how to bring it into our current workflows.
This supports actions running on multiple branches
We would need to set up the splits first, I think - once they are set up, then the action would run and push the changes when triggered.
https://github.com/splitsh/lite Symfony uses this tool but the docs are not all that great!
There is a hosted ‘service’ which uses this tool - created by one of the core Symfony team at https://www.subtreesplit.com/
Played around with it and was able to split the app/ folder from https://github.com/RCheesley/a-mautic-playground to https://github.com/RCheesley/core-lib-playground very easily.
It only pushes the default branch at the moment but the author may be able to dig into making it push other branches
It has the option to comment on PR’s that are made on the mirror repos in order to push people back to the main repo - nice feature that would hopefully save us a bit of a headache (I don’t think that you can disable PR’s on a repo)
Key point: the underlying technology seems to work!
Github Actions
https://github.com/drud/action-cross-commit this may work as a GitHub Action - we would need to work out what directories are being updated in the merged PR and push those directories to the correct repos using some kind of dynamic folder magic 🧙♀️
https://github.com/s0/git-publish-subdir-action may also be an option (we would have to set up each folder and match it with the correct repo) - maybe run an ‘if files have changed’ check beforehand to only push changes if there are changes?
...
We need to make sure that all of the plugins and themes are appropriately licensed and that a readme & license file plus all the security and community health files are present
We can set up a centralised .github repo that allows us to orchestrate all of our organisation repos centrally
We would need to reach out to the maintainers of repos ahead of time to make sure that they are aware of any changes (eg changes to branch names) that we may require / suggest
We can apply rules to groups of repos, to all repos, or manage them individually
There is a tool that we can use to manage our repos centrally: https://organizer.gitconsensus.com
Open source project https://github.com/gitconsensus/GithubOrganizer
Has a Github App https://github.com/apps/organization-manager
...