Migration to GitHub-native Dependabot: solutions for auto-merge and Action secrets
The end of Dependabot Preview is near
Dependabot Preview (dependabot.com) - which was acquired by GitHub on May 23rd, 2019 - will be shut down on August 3rd, 2021 in favor of the GitHub-native Dependabot.
One week ago, we started to see many pull requests from Dependabot across our organization's repositories:
This is very nice, an automated pull request that migrates our legacy non-native configuration file to the
new format v2. What could go wrong? This:
You have configured automerging on this repository. There is no automerging support in GitHub-native Dependabot, so these settings will not be added to the new config file. Several 3rd-party GitHub Actions and bots can replicate the automerge feature.
At first, I was like "Wtf? 😨", and of course I'm not the only one to not be happy about this change, see some GitHub issues:
- Dependabot auto merge not working
- Dependabot should automerge if all mandatory pull-request checks are passing
-
Support for
automergein GitHub Native Dependabot - "Merge on approval" on GitHub-native Dependabot
After reading all of this issues and comments, I understood this change was for security purposes and that's a good thing. However, as I commented, it was not possible for us at job:
- we have around ~200 public and private repositories,
- we are a very small team (~4 peoples),
- we cannot review, approve, and merge all Dependabot pull requests across all of our repositories, we don't have the time, and we have better things and more critical to work on,
- we have a lot of tests in our projects, and we are pretty confident for auto-merging patch and minor updates without the fear of having a non-functional project in production.
Nope, the auto-merge is gone from Dependabot, there is no checkbox Enable Dependabot auto-merge or something else to configure.
Instead, you should implement it yourself or use a 3rd party GitHub Action, which is even promoted by Dependabot (???):
Several 3rd-party GitHub Actions and bots can replicate the automerge feature.
The thing is - even if I'm an open-source contributor and really like open-source - I can't 100% trust 3rd-party GitHub Actions which use an access token with write access for merging (and auto-approving if needed), while I can 100% trust Dependabot since it's part of GitHub.
Re-enable auto-merging (with auto-approve)
EDIT: 5st june 2021
Dependabot recently released a GitHub Action dependabot/fetch-metadata which can be used to get metadata about dependencies update.
With it, you can easily know about:
- the dependency name
- the type of dependency (production or development)
- the type of update (
major,minor, ...)
For the auto-approve and auto-merge, it seems that you can use the GitHub CLI gh as described in dependabot/fetch-metadata's
README.
So, how can we re-enable auto-merging Dependabot pull requests?
The solution I've used has the following features:
- it runs automatically after our CI jobs being successful
- it respects the update type (
minor,patch...) - it can auto-approve the pull request if needed
- aaaaaand of course it auto-merge the pull request 🎉
No, I'm not using Kodiak or Renovate.
To be honest I didn't understand how to perfectly configure Kodiak to fit our needs, I felt that I could break everything with a bad configuration :sweat_smile:.
For Renovate, I was not a big fan of the GitHub App and of the self-hosted integration (which is great to configure hostRules with private auth at this level instead of doing it per project).
I'm leaving my job in one week, and I don't want to add new things that required me whole days to understand, and needs to be maintained. I don't want to leave a poisoned gift for my team. 🎁
Dependabot is doing a great job, so let's keep using it! It's fully integrated to GitHub, it's reactive, it's easily configurable and there is even a dedicated page to configure secrets for Dependabot. For example, if we need to update our packagist.com auth token, we just need to do it only once at one place, and that's fantastic.
So, given a very basic GitHub Action workflow (jokes aside, this is the kind of workflow we use at work with 2 or 3 jobs php, javascript and cypress):
1name: CI 2 3on: 4 pull_request: 5 types: [opened, synchronize, reopened, ready_for_review] 6 7env: 8 TZ: UTC 9 10 COMPOSER_ALLOW_SUPERUSER: '1' # https://getcomposer.org/doc/03-cli.md#composer-allow-superuser 11 COMPOSER_AUTH: '{"http-basic":{"repo.packagist.com":{"username":"token","password":"${{ secrets.PACKAGIST_AUTH_TOKEN }}"}}}' 12 13jobs: 14 php: 15 runs-on: ubuntu-latest 16 steps: 17 - uses: actions/checkout@v2 18 19 - uses: ./.github/actions/setup-environment 20 21 - uses: shivammathur/setup-php@v2 22 with: 23 php-version: ${{ env.PHP_VERSION }} 24 coverage: none 25 extensions: iconv, intl 26 ini-values: date.timezone=${{ env.TZ }} 27 tools: symfony 28 29 - uses: actions/setup-node@v2 30 with: 31 node-version: ${{ env.NODE_VERSION }} 32 33 - uses: actions/cache@v2 34 with: 35 path: ${{ env.COMPOSER_CACHE_DIR }} 36 key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} 37 restore-keys: ${{ runner.os }}-composer- 38 39 - uses: actions/cache@v2 40 with: 41 path: ${{ env.YARN_CACHE_DIR }} 42 key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} 43 restore-keys: ${{ runner.os }}-yarn- 44 45 # Check dependencies 46 - run: symfony composer validate 47 - run: symfony security:check 48 49 # Install environment and application 50 - run: make setup@integration 51 52 # Prepare tests 53 - run: symfony console cache:clear 54 - run: APP_ENV=test symfony console doctrine:schema:validate # force APP_ENV=test because only the test database is created 55 - run: symfony console api:swagger:export > /dev/null # Check if ApiPlatform is correctly configured 56 57 # Lint Twig, Yaml and XLIFF files 58 - run: symfony console lint:twig templates 59 - run: symfony console lint:yaml config --parse-tags 60 - run: symfony console lint:xliff translations 61 62 # Static analysis 63 - run: symfony php bin/php-cs-fixer.phar fix --verbose --diff --dry-run 64 - run: symfony php bin/phpcs 65 - run: symfony php bin/phpstan analyse 66 - run: APP_ENV=test symfony php bin/phpunit.phar # See https://github.com/symfony/symfony-docs/pull/15228 67 - run: symfony php bin/phpspec run 68 69 javascript: 70 runs-on: ubuntu-latest 71 steps: 72 - uses: actions/checkout@v2 73 74 - uses: ./.github/actions/setup-environment 75 76 - uses: shivammathur/setup-php@v2 77 with: 78 php-version: ${{ env.PHP_VERSION }} 79 coverage: none 80 extensions: iconv, intl 81 ini-values: date.timezone=${{ env.TZ }} 82 tools: symfony 83 84 - uses: actions/setup-node@v2 85 with: 86 node-version: ${{ env.NODE_VERSION }} 87 88 - uses: actions/cache@v2 89 with: 90 path: ${{ env.COMPOSER_CACHE_DIR }} 91 key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} 92 restore-keys: ${{ runner.os }}-composer- 93 94 - uses: actions/cache@v2 95 with: 96 path: ${{ env.YARN_CACHE_DIR }} 97 key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} 98 restore-keys: ${{ runner.os }}-yarn- 99 100 - run: make setup@integration 101 102 # Check TypeScript types 103 - run: yarn tsc --noEmit 104 105 # Lint JS and CSS files 106 - run: yarn lint:js --no-fix 107 - run: yarn lint:css --no-fix 108 109 # Build for developemnt and production 110 - run: yarn dev 111 - run: yarn prod 112 113 cypress: 114 runs-on: ubuntu-latest 115 name: cypress (${{ matrix.cypress.group }}) 116 strategy: 117 fail-fast: false 118 matrix: 119 cypress: 120 - group: default 121 spec: 'tests/cypress/**/*' 122 123 steps: 124 - uses: actions/checkout@v2 125 126 - uses: ./.github/actions/setup-environment 127 128 - uses: shivammathur/setup-php@v2 129 with: 130 php-version: ${{ env.PHP_VERSION }} 131 coverage: none 132 extensions: iconv, intl 133 ini-values: date.timezone=${{ env.TZ }} 134 tools: symfony 135 136 - uses: actions/setup-node@v2 137 with: 138 node-version: ${{ env.NODE_VERSION }} 139 140 - uses: actions/cache@v2 141 with: 142 path: ${{ env.COMPOSER_CACHE_DIR }} 143 key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} 144 restore-keys: ${{ runner.os }}-composer- 145 146 - uses: actions/cache@v2 147 with: 148 path: ${{ env.YARN_CACHE_DIR }} 149 key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} 150 restore-keys: ${{ runner.os }}-yarn- 151 152 - run: make setup@integration 153 154 # Start Symfony server 155 - run: APP_ENV=test symfony serve --port 8000 --daemon 156 - run: echo "CYPRESS_BASE_URL=https://localhost:8000" >> $GITHUB_ENV 157 158 - name: Run Cypress 159 if: ${{ env.IS_DEPENDABOT == 'false' && ! github.event.pull_request.draft }} 160 uses: cypress-io/github-action@v2 161 with: 162 spec: ${{ matrix.cypress.spec }} 163 record: true 164 parallel: true 165 group: ${{ matrix.cypress.group }} 166 env: 167 CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }} 168 169 - name: Run Cypress (for Dependabot or when pull request is draft) 170 if: ${{ env.IS_DEPENDABOT == 'true' || github.event.pull_request.draft }} 171 uses: cypress-io/github-action@v2 172 173 auto_approve: 174 runs-on: ubuntu-latest 175 needs: [php, javascript, cypress] 176 if: ${{ github.actor == 'dependabot[bot]' }} 177 steps: 178 - uses: hmarr/auto-approve-action@v2.0.0 179 with: 180 github-token: ${{ secrets.GITHUB_TOKEN }}
Here we have 4 jobs php, javascript, cypress and auto_approve. The last one is a bit special because it needs all the other jobs to success before running.
When jobs php, javascript and cypress will be successful, the job auto_approve will run.
We already used hmarr/auto-approve-action to automatically approve Dependabot pull requests, but it does not support auto-merge.
Instead, we now use ahmadnassri/action-dependabot-auto-merge which supports auto-approve and auto-merge Dependabot pull requests.
It can be configured through a configuration file .github/auto-merge.yml to have a more fine-grained configuration.
This is how our file looks like:
1# Documentation: https://github.com/ahmadnassri/action-dependabot-auto-merge#configuration-file-syntax 2 3- match: 4 dependency_type: development 5 update_type: semver:minor # includes patch updates! 6 7- match: 8 dependency_type: production 9 update_type: security:minor # includes patch updates!
Then we update our workflow to use the action:
1 auto_approve: 2 runs-on: ubuntu-latest 3 needs: [php, javascript, cypress] 4 if: ${{ github.actor == 'dependabot[bot]' }} 5 steps: 6 - - uses: hmarr/auto-approve-action@v2.0.0 7 - with: 8 - github-token: ${{ secrets.GITHUB_TOKEN }} 9 + - uses: ahmadnassri/action-dependabot-auto-merge@v2 10 + with: 11 + github-token: ${{ secrets.ACTION_DEPENDABOT_AUTO_MERGE_TOKEN }}
That's it! You've just successfully re-added auto-merging feature to GitHub-native Dependabot, while respecting update types, and without migrating to another new service.
The new pull requests from Dependabot will be automatically approved and merged after all your CI pass!
...
Wait what? It's red, what happens?? 💥 :rotating_light:
Share your secrets with Dependabot
If your workflows depend on Action Secrets, maybe you already faced this problem. It's a new thing from GitHub, see blog post GitHub Actions: Workflows triggered by Dependabot PRs will run with read-only permissions
Starting March 1st, 2021 workflow runs that are triggered by Dependabot from
push,pull_request,pull_request_review, orpull_request_review_commentevents will be treated as if they were opened from a repository fork. This means they will receive a read-onlyGITHUB_TOKENand will not have access to any secrets available in the repository. This will cause any workflows that attempt to write to the repository to fail.
And since we use many secrets (PACKAGIST_AUTH_TOKEN, ACTION_DEPENDABOT_AUTO_MERGE_TOKEN, ...), the workflow fails because it has no access to them.
... Sigh... another great thing from GitHub, but that for security concerns again, so it's fine I guess. What should we do to make our workflow working again?
There is already a GitHub issue Dependabot can't read secrets anymore, perfect!
After several readings, I learned about pull_request_target event which allows the workflow
to access secrets, nice!
danger
This event runs in the context of the base of the pull request, rather than in the merge commit as the pull_request event does.
This prevents executing unsafe workflow code from the head of the pull request that could alter your repository or steal any secrets
you use in your workflow. This event allows you to do things like create workflows that label and comment on pull requests based on
the contents of the event payload.
We need to be careful and only allow the Dependabot user for pull_request_target event.
How can we achieve this without duplicating our whole workflow?
- Duplicate the
on.pull_requesttoon.pull_request_targetlike this:
1on: 2 pull_request: 3 types: [opened, synchronize, reopened, ready_for_review] 4 + pull_request_target: 5 + types: [opened, synchronize, reopened, ready_for_review]
- For each job, check if
it's a pull request not opened by Dependabotora pull-request from fork opened by Dependabotlike this:
1jobs: 2 php: 3 runs-on: ubuntu-latest 4 + # If the PR is coming from a fork (pull_request_target), ensure it's opened by "dependabot[bot]". 5 + # Otherwise, clone it normally. 6 + if: | 7 + (github.event_name == 'pull_request_target' && github.actor == 'dependabot[bot]') || 8 + (github.event_name != 'pull_request_target' && github.actor != 'dependabot[bot]') 9 steps: 10 # ...
- For each job, change the way you checkout the pull request:
1jobs: 2 php: 3 runs-on: ubuntu-latest 4 # If the PR is coming from a fork (pull_request_target), ensure it's opened by "dependabot[bot]". 5 # Otherwise, clone it normally. 6 if: | 7 (github.event_name == 'pull_request_target' && github.actor == 'dependabot[bot]') || 8 (github.event_name != 'pull_request_target' && github.actor != 'dependabot[bot]') 9 steps: 10 - - uses: actions/checkout@v2 11 + - name: Checkout 12 + if: ${{ github.event_name != 'pull_request_target' }} 13 + uses: actions/checkout@v2 14 + 15 + - name: Checkout PR 16 + if: ${{ github.event_name == 'pull_request_target' }} 17 + uses: actions/checkout@v2 18 + with: 19 + ref: ${{ github.event.pull_request.head.sha }}
That's it, we updated our workflow file that supports both:
-
pull_requestevent, when you open a new pull request from the base repository -
pull_request_targetevent, for Dependabot only when it open a new pull request from a fork
After pushing your changes, Dependabot pull requests will now have access to secrets, and your checks should be green :green_heart::
Conclusion
In this article, we were able to auto-merge Dependabot pull requests again:
- we stayed with Dependabot, no migration to Kodiak, Renovate or anything else
- we added back the auto-approve and auto-merge, thanks to ahmadnassri/action-dependabot-auto-merge
- we let Dependabot access our workflow secrets again, by using
on: pull_request_targetand limiting the jobs to the Dependabot user only
Those two last days were a bit stressful, thinking of how to bring back auto-approve and auto-merge behaviours, respect the update type, if we needed to change to another dependencies manager service or not...
But that's over, I was able to get it working some hours ago, and I'm so happy!! 😄 I wanted to share my problems and solutions to the community, but also to my team to explain them what I've done those last days and what changed with Dependabot.
